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I. 



INTRODUCTION 



Joint and naval vision documents place significant emphasis on information systems and 
anticipate that the 21®' century warfighter will leverage these systems to dramatically increase 
combat effectiveness while facing a diverse and highly unpredictable enemy. Joint Vision 2010 
{JV2010) and naval warfighting concepts expressed in Operational Maneuver From the Sea 
(OMFTS) direct extensive investment in information intensive command, control, communications, 
computers, intelligence, sun/eillance, and reconnaissance (C4ISR) systems, precision weapons 
and combat support systems. Additionally, current and emerging operational concepts and 
initiatives such as Copernicus, Information Technology for the 21®' Century {iT-21), Sea Dragon, 
and Network Centric Warfare, all require robust, fault-tolerant information systems for successful 
implementation. Failure to provide the information transfer capabilities that these concepts and 
vision documents rely on will negate the positive impacts that advancements in tactics and 
weapon system technologies will provide. Therefore successful implementation of future visions 
will not be possible without a well-defined, user needs driven, communications architecture. 

Given the dispersed nature of the future battlefield it is clearly evident that an integrated 
and robust communication architecture that meets the goals set forth in 21®' century vision 
documents will rely on space based, satellite communications (SATCOM). Furthermore, 
wideband^ SATCOM assets to include commercial wideband systems will be critical contributors 
to any Department of Defense (DoD) SATCOM architecture. The high cost of these space 
systems requires a rigorous evaluation of proposed concepts before system acquisition begins. 
The cost versus capability decision should be a negotiated trade between warfighter needs and 
SATCOM system costs. Therefore, accurately defining and articulating user needs and 
performance measures that reflect these needs are critical components of this decision process. 

^ The Advanced Military Satellite Communications Capstone Requirements Document [CRD], (24 Apr 
1998 pp. 4-7), defines wideband as "High-capacity circuits and networks in excess of 64kbps - Typically in 
the multi-megabit range - Encompasses a variety of network topologies - Today principally broadcast and 
switched/full Duplex.” 
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A. GOAL 

The purpose of this research is to state the naval warfighter’s definition of value for 
commercial (COMMER) wideband SATCOM service. Doing so will assist in the selection of 
commercial wideband SATCOM service for naval forces. As Figure 1 shows the performance 
parameters presented in this thesis are derived from the "required system characteristics" as 
defined in The Advanced Military Satellite Communications Capstone Requirements Document 
[CRD], April 1998. This research is not intended to provide an all-inclusive list of performance 
parameters that uniquely define the overall system-of-systems military SATCOM (MILSATCOM) 
architecture. Instead, the intent is to clearly articulate the required performance characteristics 
that commercial (COMMER) SATCOM systems should display in order to effectively satisfy 
threshold naval wideband communications requirements. In short the objective is to define the 
characteristics of one of the components (systems) that will make up the overall system-of- 
systems MILSATCOM architecture. It is expected that this analysis will be added to and used in 
conjunction with the many other studies seeking to meet the 21®’ century naval warfighter’s 
SATCOM needs. 

PastandCurrent Militaryand 

NavalVision C o m m e rc ia I S A T C O M 



Docum ents 



L ite ra tu re 



Id e n tif ic a tio n of 
Naval Force 
C h a ra c te ris tics 



Identification of 
Wideband SATCOM 
C h a ra cteris tics 



Naval Forces* W ideband 
SATCOM Performance 
P aram eters 



Figure 1. Thesis Goal 
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B. THESIS ORGANIZATION 

This thesis consists of six chapters. A short description of each chapter is provided below. 

• Chapter I - Introduction. 

• Chapter II - Naval Force Profile. Through an analysis of relevant naval concept and 
vision documents this chapter identifies the unique characteristics that define the naval 
force profile. This profile will be integrated with the product presented in Chapter III to 
form the intended performance parameters. 

• Chapter III - Performance Parameters. Chapter III begins with a justification of the use of 
the Advanced MILSATCOM Capstone Requirement Document (CRD) as a foundation 
document for the development of performance evaluation parameters, and concludes 
with a list of the derived evaluation measures. 

• Chapter IV - Weighting and Utility Analysis. The goal of this chapter is provide a 
"strawman" evaluation hierarchy complete with weights and utility curves. A discussion of 
the user's utility for each evaluation parameter is provided as well as the process for 
deriving relative weights. 

• Chapter V - Implementation of the Results. Chapter V provides a discussion of how the 
hierarchy can be used. 

• Chapter VI - Conclusion. The conclusion provides an overview and presentation of 
related topics for follow-on research. 

• Appendix A - Cost and Cost As An Independent Variable. This appendix presents the 
author's opinion on how affordability and cost as an independent variable should be 
included in the selection of wideband COMMER SATCOM. 

• Appendix B - System Engineering and Evaluation Regime (SEER). This appendix is a 
paper written by employees of The Aerospace Corporation and Naval Space Command 
and it describes a decision support tool called SEER. 
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II. 



NAVAL FORCE PROFILE 



Despite streamlining introduced by DoD acquisition reform, lead-time for acquisition of 
complex systems is often still long enough to cause the initial requirements to become outdated 
prior to system fielding. This problem is further compounded for C4ISR systems, which are 
subject to the rapid technology innovation cycle that currently exists in this industry. Efforts at 
reducing a system’s development timeline should be coupled with a well-defined requirements 
definition process that includes the creation of a user profile. Characteristics that describe and 
define a user profile are extremely valuable for making educated predictions about future 
requirements. Given defined requirements and an accurate user profile, developers can design an 
open system that allows for expansion and refinement along the bounds of the user’s profile. 
Simply stated, identification of user characteristics allow for the creation of a user profile which 
leads to the development of a system that can flex to meet unanticipated requirements or take 
advantage of emerging technologies. The intent of this chapter is to use DoD and naval doctrine 
and concept publications to identify naval force characteristics and create a naval force profile. 
This profile will be integrated with the product presented in Chapter III to form the intended 
performance parameters. 

A. APPLICABLE CONCEPTS AND VISIONS 

Two comprehensive 1997 studies, the Quadrennial Defense Review {QDR) and the 
National Defense Panel (NDF) “both characterize the next century as one of crisis, conflict, and 
chaos in the littorals brought on by rapid economic growrth, increased competition for limited 
resources, terrorism, technological diffusion, exponential growrth in urban populations, 
nationalism, ethnic and religious strife, and increasing access to modern conventional weaponry 
and weapons of mass destruction.” [Krulak, 1998, pp.16] These studies reflect an evolution 
(shown in Figure 2) of thought initially “articulated by the President at the Aspen Institute on 
August 2, 1990.” [...From the Sea, 1996] The themes of the August 1990 address were 
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documented in 1992 with the President’s publication of the National Security Strategy {NSS) and 
the Department of the Navy’s (DoN) publication of ...From the Sea. 



1992 

National Security Strategy 
...From the Sea 

1994 

Forward... From the Sea 

1996 

Joint Vision 2010 

Operational Maneuver from the Sea 
Naval Operational Concept 

1997 

National Security Strategy for A New Century 
Quadrennial Defense Review 

Concept for Future Joint Operations, Expanding Joint Vision 2010 

1998 

Forward... From the Sea Anytime, Anywhere 
2f^ Century 



Figure 2. Applicable Concept and Vision Documents 

The NSS of 1992 and the DoN response, ...From the Sea, documented the DoD’s 
departure from the previously held strategic vision driven by a bi-polar (United States vs. the 
Soviet Union) world. The fundamental change in policy reflected a shift “from a focus on a global 
threat to a focus on regional challenges and opportunities”. [..From the Sea, 1992] As described in 
...From the Sea this shift in focus implied a redirection and expansion of the traditional 
expeditionary role of naval forces. Although, naval forces were considered to be full participants in 
each of the principal elements of the 1992 NSS (strategic deterrence and defense, forward 
presence, crisis response and reconstitution), naval forces were identified as the primary 
providers of the forward presence and crisis response elements. [...From the Sea, 1992] Naval 
leadership made a clear association between the Force’s expeditionary mindset and the NSS 
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elements of forward presence and crisis response. Naval expeditionary forces are capable of 
conducting a wide range of operations (port visits to major offensives), from sea without host 
nation support. In short “Naval Expeditionary Forces provide unobtrusive forward presence which 
may be intensified or withdrawn as required on short notice.” [...From the Sea, 1992] Having 
demonstrated that naval forces were critical to the execution of the NSS ...From the Sea identified 
four key operational capabilities that were required to successfully execute the new direction of 
the Navy and Marine Corps: 

• Command, Control, and Surveillance 

• Battlespace Dominance 

• Power Projection 

• Force Sustainment 

Evolution of ...From the Sea was provided in 1994 with the publication of Forward... From 
the Sea. Forward... From the Sea expands on the previous shift in focus and continued the 
redirection of the “Naval Service away from operations on the sea, toward power projection and 
the employment of naval forces from the sea to influence events in the littoral regions of the world” 
[Forward... From the Sea, 1994] In addition to reinforcing the expeditionary nature of the naval 
forces, the 1994 document identified the Aircraft Carrier Battle Groups (CVBG) and the 
Amphibious Ready Groups (ARG) as the “basic presence ‘building blocks’” that are fully capable 
of meeting the NSS’s requirement for versatile, multipurpose forward forces. Finally the document 
concluded with an expansion of the previously mentioned four key operational capabilities to five 
fundamental and enduring naval force roles. These roles were: 

• Projection of power from sea to land 

• Sea control and maritime supremacy 

• Strategic deterrence 

• Strategic sealift 

• Forward naval presence 
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1996 saw the publication of three documents that further promoted and defined the role 
that naval forces would play in the 21®' century. These documents were, Joint Vision 2010 
(JV2010), Operational Maneuver From the Sea (OMFTS), and the Navy Operational Concept 
(NOC). The three documents are highly complementary and are intended to be digested together 
as an integrated concept. JV2010 serves as a strategic guidebook that seeks to channel the 
individual Services’ "distinctive capabilities, cultures, and traditions" towards the ultimate goal of 
effective joint warfighting. [Krulak, 1998, pp. 18] To do this, JV2010 provides the Services with 
four operational pillars, (dominant maneuver, precision engagement, focused logistics, and force 
protection) which when tied together by a robust and superior information technology architecture 
leads to (as described by JV2010) full spectrum dominance. 

OMFTS and NOC are the Marine Corps’ and Navy’s description of how naval forces will 
execute their missions in accordance with JV2010 and more importantly the NSS. Both 
documents are entirely compatible with JV2010’s operational themes and continue the emphasis 
on littoral operations originally presented in ...From the Sea and Forward... From the Sea. The 
fundamental premise behind OMFTS and NOC is the need to maintain a naval expeditionary force 
equipped and ready to meet the challenges of the 21®* century. The following quote more fully 
describes this notion. 

To influence events overseas, America requires a credible, forwardly deployable, 
power projection capability. In the absence of an adjacent land base, a 
sustainable forcible entry capability that is independent of forward staging bases, 
friendly borders, overflight rights, and other politically dependent support can 
come only from the sea. The chaos of the future requires that we maintain the 
capability to project power ashore against all forces of resistance, ranging from 
overcoming devastated infrastructure to assisting a friendly people in need of 
disaster relief to countering the entire spectrum of armed threats. [OMFTS, 1996] 

OMFTS and NOC are closely coupled interdependent concepts. The critical difference 
between OMFTS and historical (or traditional) operational maneuver is the "extensive use of the 
sea as a means of gaining advantage, an avenue of friendly movement that is simultaneously a 
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barrier to the enemy and a means of avoiding disadvantageous engagements." [OMFTS, 1996] 
The ability to rely on the sea as an avenue for movement depends on a force’s ability to maintain 
maritime superiority. The primary tenets set forth in the NOC call for the ability to gain and 
maintain control of sea lanes, ocean areas critical to the success of the overall national strategy, 
and operational sea control of contested littoral areas. [Copernicus, 1997] Therefore, NOC as an 
operational concept directs naval forces to shape and dominate the littoral battlespace to ensure 
maritime superiority for naval and joint force operations. Assured that maritime superiority is 
gained and maintained OMFTS seeks to "combine doctrine with technological advances in 
mobility, fire support, communications and navigation to rapidly identify and exploit enemy 
weaknesses across the entire spectrum of conflict.” [Copernicus, 1997] Critical to OMFTS is the 
emergence of new technologies that will allow leaner, yet more capable forces to project power 
from sea-based platforms without the creation of the traditional shore-based logistics and staging 
areas. [OMFTS, 1996] Following this line of thought OMFTS provides six principles that clearly 
articulate the essence of the concept. These principles are: 

• O/WFTS focuses on an operational objective. 

• OMFTS uses the sea as maneuver space. 

• OMFTS generates overwhelming tempo and momentum. 

• OMFTS pits strength against weakness. 

• OMFTS emphasizes intelligence, deceptions, -and flexibility. 

• OMFTS integrates all organic, joint, and combined assets. 

May 1997 saw the publication of three high level documents that link naval force 
characteristics and the United States’ defense policy. The QDR and the President’s publication of 
A National Security Strategy for A New Century (NSS 1997) provides high level guidance to the 
Armed Services. While the Joint Chiefs of Staff publication of Concept for Future Joint 
Operations, Expanding Joint Vision 2010 (CJFO) provides an increased level of detail and 
explanation of JV2010. 
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The cornerstone theme of NSS 7997 is “the imperative of engagement.” [NSS 1997] The 
“strategy of engagement presumes the United States will continue to exercise strong leadership in 
the international community, using all dimensions of its influence to shape the international 
security environment.” [QDR, 1997] The President groups threats to the United States into three 
broad intertwined categories: (1) regional or state-centered threats, (2) transnational threats, and 
(3) threats from weapons of mass destruction. [NSS 1997] These threats are to be countered 
using a three-prong engagement centered policy. The first prong directs U.S. forces to shape the 
international environment, number 2 requires the Armed Forces to “maintain the capability to 
respond to the full spectrum of threats”, and the third and final prong instructs the Services to 
“prepare now for the threats and dangers of tomorrow and beyond.” [QDR, 1997] Although NSS 
1997 does not task naval forces to execute specific operations it is very important in that it defines 
the strategic boundaries that naval leadership will use to guide the Navy and Marine Corps into 
the 21®‘ Century. 

Published in the same month and year as NSS 1997 the QDR presents the DoD’s plan for 
implementation of NSS 1997. In short the QDR documents the implementation options that the 
Armed Services analyzed during their search to define an overarching defense strategy. In 
concert with NSS 7997 the QD/T emphasizes the “shape- respond-prepare strategy.” [QDR, 1997] 
Critical to the shape and respond elements is the DoD’s ability to maintain a continuous overseas 
presence and the “ability to respond to a variety of smaller scale contingencies and asymmetric 
threats.” [QDR, 1997] Furthermore, the QDR recognizes that preparation for future threats, who 
will also acquire new capabilities, is critical to the success of all US forces. Embracing the 
concepts presented in JV2010 the QDR places notable emphasis on the development, fielding 
and use of C4ISR systems. Finally the QDR states that the “U.S. military must be a capabilities- 
based force that gives the national leadership a range of viable options for promoting and 
protecting U.S. interests in peacetime, crisis, and war.” [QDR, 1997] A force capable of this type 
of full-spectrum dominance must possess a “balanced mix of overseas presence and power 
projection capabilities.” [QDR, 1997] Critical enablers of this balanced force are quality people, a 
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“globally vigilant intelligence system,” a global communications architecture, superiority in space, 
and control of the seas and airspace.” [QDR , 1997] 

Also published in May of 1 997 CJFO further outlines how the DoD intends to execute the 
shape-respond-prepare strategy. Evolutionary in nature the primary purpose of CJFO seems to 
be to better articulate and describe the concepts originally presented in JV2010. The document 
expands and clarifies the four JV2010 operational concepts (dominant maneuver, precision 
engagement, full-dimensional protection, focused logistics) and how these concepts coupled with 
the increased use of information technology will be used to achieved full spectrum dominance. An 
important addition to CJFO that was not as well presented in JV2010 is a chapter-long discussion 
of information superiority and its relationship to the four operational concepts. Information 
superiority is defined as ‘1he capability to collect, process, and disseminate an uninterrupted flow 
of information while exploiting or denying an adversary’s ability to do the same.” [JV2010, 1996] It 
consists of three components: 

• Information systems - which collect, process, and disseminate information 

• Information operations - the exploiting and denying of information 

• Relevant information - full range of pertinent uninterrupted flow of information. [CJFO, 
1998] 

The fundamental premise that both JV2010 and CJFO provide about information superiority is 
that it is the critical enabler of the four operational concepts and therefore crucial to the successful 
execution of the shape-respond-prepare strategy. 

Collectively the QDR, NSS 1997, and CJFO present a unified interdependent strategy for 
the defense of the nation and progression into the future. The most recent naval response to 
these documents, which is also definitely a continuation and refinement of the transformation 
initiated in 1992, is the 1998 DoN Posture Statement, Forward... From the Sea Anytime, 
Anywhere. This 71 page publication articulates what roles the Navy and Marine Corps will perform 
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in the execution of the national shape-respond-prepare strategy. Forward... From the Sea 
Anytime, Anywhere, skillfully and accurately states that: 

Shaping and responding require presence -- maintaining forward-deployed 

combat-ready forces. Being ‘on-scene’ matters! It is and will remain a distinctly 

naval contribution to peacetime engagement. 

It is plainly obvious that naval forces are uniquely tied to the nation’s overall strategy of 
engagement. Not only do naval forces provide peacetime engagement and deterrence but by 
likening naval forces to a temperature control rheostat naval leadership states that these forces 
provide the National Command Authorities (NCA) with an adjustable response capability. 
[Forward. ..From the Sea Anytime, Anywhere, 1998] ‘The flexible, rapid movement of naval 
forces at the onset of any crisis is an ideal way to signal our nation’s commitment.” 
[Forward... From the Sea Anytime, Anywhere, 1998] Furthermore, the inherent self-sufficiency 
and depth of capabilities that naval forces possess provides the NCA a range of balanced and 
flexible options. Once stating this central precept Forward.. .From the Sea Anytime, Anywhere 
proceeds with a review of the Navy and Marine Corps’ performance during 1997 presenting past 
operations as examples of doctrine in action. As evidence that the Navy and Marine Corps are 
actually pursuing the themes that have been put forth the document concludes with an extensive 
presentation of how the two Services will carry out the prepare portion of NSS 1997. Significant 
emphasis is given to recruitment, retention and education of a quality work force as well as a 
detailed discussion of each Services’ major equipment modernization programs. 

It is important to note that each of the Services are using experimental methods to ensure 
that innovative technologies and techniques are evaluated. The Marine Corps Warfighting Lab 
has developed a three phase, five year experimentation plan, commonly named Sea Dragon, that 
provides the venue for conducting “concept-based experimentation and the introduction of 
science and technology into the operating forces.” [Forward... From the Sea Anytime, Anywhere, 
1 998] The Navy employs fleet units as At-Sea Battle Labs and has initiated a series of fleet battle 
experiments (FBE) that use operational forces to ‘lake forward-looking programs and integrate 
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them with innovative operational concepts.” [Forward... From the Sea Anytime, Anywhere, 1998] 
As stated “these experiments focus on future programs that align the Navy with Joint Vision 2010 
and other emerging operational concepts.” [Forward... From the Sea Anytime, Anywhere, 1998] 
One of the newest operational concepts that will significantly impact future naval warfare is the 
emergence of network centric warfare. Unlike traditional platform-centric warfare network-centric 
warfare; 



Derives its power from the networking of a well-informed but geographically- 
dispersed force. The enabling elements are a highly-webbed information service, 
access to all appropriate information sources, weapons reach with precision and 
speed of response, value-adding command-and-control processes - to include 
high-speed automated assignment of resources to need - and integrated 
sensors hosted on the information network and closely coupled in time to the 
shooters and command-and-control processes. Network-centric warfare is 
applicable to all levels of warfare and contributes to the coalescence of strategy, 
operations, and tactics. It is transparent to mission, force size, and composition, 
and geography. [Forward. ..From the Sea Anytime, Anywhere] 



Another emerging concept is the notion of complex adaptive systems (CAS) and treating 
warfare as a CAS. The topic is discussed and related to warfare in Marine Corps Doctrinal 
Publication 1-1 Strategy, (MCDP 1-1) and the upcoming Marine Corps publication. Maneuver 
Warfare Science 1998. Although a detailed discussion of CAS is not appropriate for this 
document a brief discussion of the paradigm is applicable. Often labeled complexity theory, the 
fundamental precepts recognize that the world is not "inherently mechanical, predictable, and 
rational" instead the world is "flowing, adapting, nonlinear and holistic." [Upton, 1998, pp.1] 
Furthermore, complex systems are inherently dynamic and unpredictable making it nearly 
impossible to definitively predict the outcome of applied external forces. [Bassford, 1998, pp.1 4- 
15] Equating warfare, that is the imposition of external forces on an enemy, with complexity 
theory requires current military leaders to shift from their classical desire to impose order on 
disorder and "live within chaos." That is, since complex systems are dynamic and unpredictable 
the strategist must be prepared to adapt and exploit sudden opportunities or recover from 
unexpected setbacks. [Bassford, 1998, pp.1 7] Although still in its infancy, and far from being 
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accepted or understood by the vast majority of naval warfighters complexity theory has the 
potential to revolutionize the way in which warfare is conducted. At a minimum the notions of 
complexity and the idea that "military strategy demands a capacity for both painstaking planning 
and energetic adaptation to unfolding events" are certainly applicable to the NSS 1997 shape- 
respond-prepare strategy. [Bassford, 1998, pp.17] Therefore, it is vital that the critical elements of 
network-centric warfare and complex adaptive systems theory are included in any characterization 
of naval forces. 

B. NAVAL FORCE CHARACTERISTICS 

Forecasting future requirements so that developed systems are flexible enough to 
accommodate growth is more accurate when an established theme or trend is available. Such a 
trend is clearly visible in the series of doctrine and concept publications presented in the previous 
section. This shared central theme assists in identifying the common characteristics that make up 
a user’s profile and therefore aids in describing the future requirements vector. More simply 
stated, a trend has identifying characteristics, a collection of characteristics creates a profile, a 
profile can be used to forecast future needs. 

It is evident that a central theme of each document is that naval forces are critical to the 
successful execution of NSS 1997. Due to their ability to provide powerful yet unobtrusive 
presence naval forces are uniquely capable of executing the shape and respond elements of the 
shape-respond-prepare strategy. Furthermore, "the purpose of naval forces is to influence events 
ashore directly and decisively from the sea ...anytime, anywhere." [Forward... From the Sea 
Anytime, Anywhere, 1998] Given the breadth of these statements it is important to further 
delineate the characteristics of our U.S. naval forces. Naval forces are [...From the Sea, 1992]: 

• Expeditionary nature - which implies a mind set, a culture, and a commitment to forces 
that are designed to operate forward and to respond swiftly in austere environments. 

• Structured to project power - from the sea when required by national demands. 

• Self-supporting - self-sufficient forces able to sustain support for long-term operations in 
remote areas of the world. 
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• Unrestricted - not restricted by the need for transit or over-flight approval form foreign 
governments in order to enter the scene of action. 

• Scalable - capable of withdrawing or intensifying operations on short notice. 



The following text and Figure 3. Naval Force Profile, clarifies these characteristics further: 

• Expeditionary nature, capable of: 

• Operating forward - implies operating regionally as leaner dispersed forces away 
from infrastructure - capable of a wide range of missions - closely associated with 
self-sufficiency. 

• Responding swiftly - timeliness is a critical component of success and if required 
lethality, highly mobile forces “rapidly assembled for the mission at hand and able to 
be rapidly extracted or relocated as the mission is completed” [Sea Dragon, 1997] - a 
pr/on location information is a luxury - reduced planning time requires flexibility 
across all warfare disciplines (C^, aviation, ground, sea). 

• Structured to project power - all components of the force contribute to the force's 
overall capability to conduct “coordinated and effective operations against a wide range of 
enemy targets, facilities, and infrastructure over a larger area of action (AOA).“ [Sea 
Dragon, 1997]. - includes the use of long-range precision firepower and is closely 
coupled with the concepts of sea-based support - implies the ability to unilaterally conduct 
forcible entry from the sea, that is “project power ashore in the face of armed opposition" 
without the use of established logistics lodgments on the beach. [Krulak, 1998, pp.19] 

• Self-supporting - self-sufficient - independent forces capable of acting without the need 
of host nation support or the buildup of ashore staging facilities - capable of executing 
sea-based support and logistics - logistics on demand - high responsiveness and 
reduced vulnerability (no ashore staging area to protect) - implies highly durable and easy 
to maintain end items. 

• Unrestricted - Not limited by accepted international regulations - strikes at the heart of a 
naval force's ability to operate and maintain an effective presence in accordance with 
accepted international boundaries - implies the use of standoff and over the horizon 
weapon and transport systems not requiring the permission of neutral or passively hostile 
countries for over-flight approval in order to enter the region of contention - closely 
related to self-sufficiency in that there is an implied capability of self-reliance requiring little 
to no need for use of “in-country" infrastructure. 

• Scalable - NCA rheostat - implies the ability to expeditiously size forces to meet the 
strategic need - suggests the capacity to conduct a range of intervention options, from a 
show of force to humanitarian assistance to full scale war - indicates the capability to 
conduct covert (i.e. submerged submarine operations) or overt (i.e. CVGB steaming in 
view of the coast) presence missions - the ability to provide a credible presence 
unilaterally without the approval or support of the international community. 
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1992 

National Security Strategy 

* Regional vs. Global Threat * Strategic Deterrence and Defense * 

* Forward Presence * Crisis Response * Reconstitution * 

...From the Sea 

* Naval Expeditionary Forces * Command, Control, and Surveillance * 

* Battlespace Dominance * Power Projection * Force Sustainment * 

1994 

Forward.. .From the Sea 

* CVBG and ARG are the building blocks * Projection of Power from Sea to Land * 

* Sea Control and Maritime Supremacy * Strategic Deterrence * Strategic Sealift * 

* Forward Naval Presence * 

1996 
JV2010 

* Dominant Maneuver * Precision Engagement * 

* Full Dimensional Protection * Focused Logistics * Information Superiority * 

OMFTS 

* Regional vs. Global Threat * Strategic Deterrence and Defense * 

* Forward Presence * Crisis Response * Reconstitution * 

Navy Operational Concept 

* Establish and Maintain Maritime Superiority * Control Sea Lanes and Ocean Areas * 
* Establish Operational Sea Control of Littoral Areas * 

1997 

National Security Strategy for A New Century 
* Engagement * Shape * Respond * Prepare * 

QDR 

* Shape * Respond * Prepare * 

CFJO 

* Dominant Maneuver * Precision Engagement * 

* Full Dimensional Protection * Focused Logistics * Information Superiority * 

1998 

Froward.. .From the Sea Anytime, Anywhere 
* NCA Rheostat * Forward Deployed * 

* Shape * Respond * Prepare * 

2f‘ Century 
Naval Force Profile 
* Expeditionary Nature * 

* Structured To Project Power* 

* Self-supporting * 

* Unrestricted * 

* Scalable * 



Figure 3. Naval Force Profile 
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Finally, despite the numerous documents and writings describing naval warfare and naval 
forces, ...From the Sea, the original publication that initiated the transformation of the U.S. naval 
forces provides the most accurate definition of these forces. 

American Naval Forces provide a powerful yet unobtrusive presence; strategic 
deterrence: control of the seas; extended and continues on-scene crisis 
response; project precise power from the sea; and provide sealift if larger scale 
warfighting scenarios emerge. These maritime capabilities are particularly well 
tailored for the forward presence and crisis response missions articulated in the 
President’s National Security Strategy. [...From the Sea, 1992] 

The integration required to mix these warfare tasks into a cohesive fighting strategy 
implies the use of advanced sensors, portable real-time command and control systems, all-source 
real-time situational awareness systems and a fault tolerant robust communications architecture. 
This communications architecture must not encumber or negate any of the naval force’s positive 
characteristics, instead it must be an efficient multiplier acting to exponentially enhance naval 
force effectiveness. COMMER wideband SATCOM systems will be crucial components of such an 
architecture. Relating the naval force profile to COMMER wideband SATCOM systems is the 
purpose of the next chapter. 

C. WHAT IS WIDEBAND AND WHY DO NAVAL FORCES NEED IT? 

1. What is Wideband? 

The DoD definition of wideband as taken from the CRD is ""High-capacity circuits and 
networks in excess of 64kbps - Typically in the multi-megabit range - Encompasses a variety of 
network topologies - Today principally broadcast and switched/full Duplex.” [CRD, 1998, pp. 4-7] 
Further characterization of wideband generically includes: none to low levels of protection; 
capable of handling data rates for transmission of bundled voice circuits, large data files, and 
video: and typically operating in the super high frequency bands (SHF) of C, X, Ku and Ka, with 
future commercial expansion to the extremely high frequency (EHF) V band. [Finnegan, 1997] 
Wideband SATCOM is also either simplex (receive only, broadcast) or duplex (user has transmit 
and receive capability). Commercially the spacecraft industry refers to wideband systems as 
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broadband systems, however neither of the two terms have clearly stated definitions. In either 
case both wideband and broadband, are used to refer to SATCOM systems that possess a 
communications capacity equivalent to terrestrial fiber optic cable. 

2. Why is it Needed? 

There are numerous needs that drive the naval forces’ requirement for wideband 
SATCOM. Most of them center on the requirement to transmit high volumes of information 
between users and producers in a real time tactically significant manner. Typical uses of 
wideband SATCOM include: imagery transfer, collaborative planing, video teleconferencing, 
command and control systems exchange, tele-medicine, trunked telephone lines, and large file 
transfer (e.g., the air tasking order). [Boyd, 1997] Furthermore, as naval forces continue to shrink 
in size naval leadership must leverage C4ISR systems to enhance individual unit situational 
awareness and operational effectiveness. Without wideband SATCOM future C4ISR systems will 
not function. The bottom line is that wideband communications are required to ensure that the 21®* 
century naval warfighter is adequately prepared to fight his enemy. Detailed listings and 
descriptions of naval SATCOM needs are contained in the Integrated Communications Data Base 
(ICDB) and the Emerging Requirements Data Base (ERDB). The ICDB lists Chairman of the 
Joint Chiefs of Staff (CJCS) validated and approved MILSATCOM requirements and is revalidated 
every two years. The ERDB is a "unit based compilation of future SATCOM requirements" and is 
used in the "planning and sizing of future MILSATCOM architectures." [Finnegan, 1997] 

Deciding what the future MILSATCOM architecture should consist of is an ongoing DoD 
struggle. Although not the complete solution, commercial wideband SATCOM is seen as having 
the capability to augment a DoD owned and operated wideband architecture. [SPACE NEWS, 
1998, pp. 8] Naval forces have already seen the effective use of commercial wideband in its 
Challenge Athena Program and in the joint service use of Phase 1 Global Broadcast Service. 
Furthermore, under the Information Technology for the 21®* Century initiative {IT-21) Inmarsat B 
will be used to provide 64kbps to deployed navy ships beginning in late 1998. [Hampton, 1997, 
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pp.8] Clearly given the history of commercial use and the explosion and innovation that is ongoing 
in the commercial SATCOM industry continued and increased future naval use of commercial 
wideband is a reality. Deciding whether that use will augment or meet all naval MILSATCOM 
wideband requirements is not the purpose of this research. Instead the goal is provide a format for 
articulating naval needs so that source selection of commercial wideband service provider’s is 
executed in a manner that best satisfies our naval warfighter’s requirements. 
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III. 



PERFORMANCE PARAMETERS 



A. OVERVIEW 

The ultimate goal of this research is to present the naval warfighter’s definition of value for 
COMMER wideband SATCOM service. Doing so will provide a means for comparing different 
COMMER wideband SATCOM systems and identifying which one best satisfies naval force 
needs. As presented in Chapter II this research proposes that naval force needs be represented 
by a user profile, composed of a set of user characteristics (expeditionary nature, structured to 
project power, self-supporting, unrestricted and scalable). These characteristics identify U.S. 
naval forces as unique contributors to the national security strategy. Use of an inadequate 
communications system will inhibit the synergistic interaction of the characteristics and degrade 
the naval force’s ability to execute its portion of the national security strategy. Therefore, the top- 
level question that this research seeks to answer is: 

Does a particular COMMER wideband SATCOM alternative complement the 
characteristics unique to U.S. Naval Forces? 

"Complement the characteristics" is used instead of "meeting the needs" to indicate that 
information systems are force effectiveness multipliers, and naval force effectiveness can be 
improved by their use. 

In order to conduct a thorough analysis and correctly answer the top-level question, naval 
force characteristics must be connected to SATCOM terminology. In short, naval needs must be 
equated to satellite and satellite constellation design (capability) parameters. Matching design 
parameters with user needs is best performed using a hierarchical organization of evaluation 
measures. This research uses a three tiered hierarchy. The highest level of the hierarchy 
represents user mission needs with the goal of selecting the best alternative. The middle level 
represents the functional objectives, "which are statements describing the tasks that a 
communications system is expected to perform." [MILSATCOM Polar Adjunct, 1993, pp.3] 
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Functional objectives “address specific goals that collectively satisfy the mission needs." 
[MILSATCOM Polar Adjunct, 1993, pp.3] The lowest level consists of individual attributes grouped 
under the functional objective which they define. These attributes are the initial nodes that link 
system design parameters to user characteristics. The next section. Defining the Process, and 
Figure 4. Hierarchical Organization of Evaluation Measures, describes the effectiveness hierarchy 
in greater detail. 



B. DEFINING THE PROCESS 



System Effectiveness Value 

Measures the degree to which all the fcrictional objectives are met 
The goal is to use System Effectiveness Value (SEV) to select the best alternative. 

SEV = 2(wi*MOEi) 



Functional Objectives 

Are tasks that the communication system is expected to perform. 

There are 7 of them and each or*e is reflected numerically as a measure of effectiveness (MOE). 
MOEs measure the degree to which a Functional Objective is attained and MOE = lE(wj*MOPj) 



Measures of Performance 

Are attributes that identify and define a functional objective (MOE). 

The number of MOPs varies for each MOE. 

MOPs measure the degree to which a critcal characteristb of a functional objective is attained. 



Figure 4. Hierarchical Organization of Evaluation Measures 
1. Level 1: System Effectiveness Value 

The top-level question in this context is somewhat analogous to the idea of a single 
mission needs statement. It is a consolidated statement that attempts to wrap-up user 
requirements in one refined sentence and provides direction to the entire selection process. For 
this research the mission needs statement, or top-level statement represents naval force needs in 
the form of a question. 
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Does a particular COMMER wideband SATCOM alternative complement the 
characteristics unique to U.S. Naval Forces? 

Satisfactory answers (acceptable alternatives) to this question fulfill naval user needs. 
Results of alternative analysis are presented as a numerical score, the System Effectiveness 
Value (SEV). SEV is a weighted sum of all the functional objectives (measures of effectiveness 
(MOE)) and reflects the degree to which all the functional objectives are attained; 

• SEV =Z(Wi * MOEi) 

• W| = the weight of the i*^ MOE 

• MOEi = the i"^ MOE that reflects the i'^ functional objective 

The SEV is a relative (ordinal) score. That is, if System A receives an SEV of 50%, it does 
not indicate that 50% of naval needs are met; instead it implies that System A meets less needs 
than System B which scored 75%. However, since the user’s utility for alternative effectiveness is 
non-linear it does not imply that System B is 25% more effective than System A. Utility is a means 
for representing how much value a user has for a particular level of performance (SEV score). A 
detailed discussion of utility theory and its use in this research is provided in Chapter IV . 

2. Level 2: Functional Objectives 

Functional objectives make up the middle level of the analysis hierarchy. Reported 
numerically as measures of effectiveness (MOE) they reflect the degree to which a functional 
objective is attained. This research uses seven functional objectives taken from the U.S. Space 
Command’s, Advanced Military Satellite Communications Capstone Requirements Document 
(CRD). Defined by the CRD as required characteristics, and renamed functional objectives in this 
research, these seven items are; coverage, capacity, protection, access and control, 
interoperability, flexibility, quality of service. Individual clarification and discussion of these seven 
functional objectives is the focus of a later section in this chapter. 

Similar to the top-level question, each functional objective has a user needs derived 
definition. Furthermore, this definition relates a SATCOM system’s (user terminal, satellite and/or 
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satellite constellation) design characteristics to user needs. Functional objectives translate user 
needs into system capabilities and design parameters. Just as the SEV reflects the weighted 
aggregation of MOEs, MOEs are the numerical result of the weighted sum of measures of 
performance (MOP). 

• MOEi=I(Wj*MOPj) 

• Wj = the weight of the j’^ MOP 

• MOPj = the j’^ MOP for the i’^ functional objective (MOE) 

Lastly, the CRD also presents affordability as an eighth consideration. "Although not 
related directly to technical performance of an individual system, affordability is a fundamental 
overarching consideration in cost constrained times." [CRD, 1998, pp. 4-1] This thesis focuses on 
the technical parameters however a discussion of affordability in terms of a presentation of a point 
of view is provided in Appendix A. Cost and Cost As An Independent Variable. 

3. Level 3: Measures of Performance 

Measures of performance are the lowest tier in the hierarchy. Each MOP is grouped 
under a functional objective and directly relates a user need with a particular SATCOM capability. 
Taken as a group the MOPs for a given functional objective define that objective. In essence, 
MOPs restate user needs as specific SATCOM capabilities. Although not written to the level of a 
traditional "detailed specification" MOPs should be viewed as the top level specifications that a 
user desires a wideband SATCOM system to have. 

There is no set number of MOPs per functional objective; some have more or less than 
others. Although the desire is to fully characterize a functional objective, realistically this can not 
be performed without an exceedingly large number of performance measures. The intent is to 
identify MOPs that provide critical links between user needs and system design. Therefore, the 
desire is to provide a useable list (i.e. not to long, not to short) of critical indicators (MOPs) that 
adequately discriminate between competing system capabilities. 
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Each MOP is weighted relative to the other MOPs within a particular functional objective 
and the weighted sum of MOPs equals that functional objective’s MOE. Since a user’s attitude 
towards value (risk) may be non-linear over his range of options (non-linear utility) the weighted 
sum of MOPs results in a collection of ordinal MOEs. Therefore, attempts to state that System A 
scores 25% lower than System B for the coverage functional objective are misleading. All that can 
be stated is that System A is ranked lower than System B for coverage. The analyst must refer to 
the functional objective’s (MOE) utility curve to discern the value of the marginal difference 
between System A and System B. As previously stated utility theory will be discussed in greater 
detail in Chapter IV . 

4. Summary 

The bullet list below and Figure 5, summarize the evaluation hierarchy. 
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Figure 5. System Evaluation Hierarchy 



• 1®* Level: System Effectiveness Value 

• SEV represents the overall goal of the decision process; selecting the optimum 
alternative. 

• SEV answers the top-level question; Does a particular COMMER wideband 
SATCOM alternative complement the characteristics unique to U.S. Naval 
Forces? 
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• The top-level question is analogous to a mission needs statement and concisely 
reflects the naval force characteristics; expeditionary nature, structured to project 
power, self-supporting, unrestricted, and scalable. 

• SEVs are ordinal, and are generated using SEV =Z(W| ’ MOEj). 

• 2"“ Level: Functional Objectives (MOEs) 

• There are seven functional objectives; coverage, capacity, protection, access and 
control, interoperability, flexibility, quality of service. 

• They are reflected numerically as MOEs where, MOEj =I(Wj * MOP,). 

• Level: Measures of Performance (MOPs) 

• The lowest level of the hierarchy. 

• Directly relate a user need to a SATCOM attribute. 

• Taken together the MOPs for a given functional objective define that objective. 

C. THE CRD AND REQUIRED SYSTEM CHARACTERISTICS 

1. What is the CRD, and What Good is it? 

The Advanced Military Satellite Communications Capstone Requirements Document 
(CRD) published by U.S. Space Command is intended as a resource that “is intermediate in 
scope and detail between MNS (mission needs statement) and an ORD (operational requirements 
document)." [CRD, 1998, pp.1-3] In April 1996 the Joint Requirements Oversight Council (JROC) 
approved the MILSATCOM Follow-on Mission Needs Statement (MNS), which addresses DoD’s 
need for increased satellite communication capabilities and capacity. [CRD, 1998, pp.1-2] As a 
high level document the MNS does not adequately articulate DoD MILSATCOM^ requirements 
and U.S. Space Command was tasked to author "an over-arching requirements document 
addressing the MILSATCOM user community’s requirements for the future DoD space, terminal, 
and control systems." [CRD, 1998, pp.1-3] The CRD is intended to be a living document that will 

2 MILSATCOM; Defined by the CRD, "as Military Satellite Communications, encompassing all 
types of SATCOM systems and services used by DoD - both DoD-owned and operated SATCOM 
systems, and DoD’s use of commercial SATCOM services." 
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be updated and kept current as ORDs for components of the system-of-systems MILSATCOM 
architecture are approved and ultimately programmed for. The document defines top-level 
characteristics for the overall system-of-system MILSATCOM architecture. These characteristics 
were vetted through a group of senior officials called the Senior Warfighter’s Forum (SWarF) and 
ultimately approved and validated by the JROC in April of 1998. As a JROC validated document 
the Cf?D will not only guide the development of future MILSATCOM ORDs but it "will also serve as 
the measure to judge how well those ORDs and plans collectively are doing in fulfilling the tenets 
of the architecture and its associated programmatic course of action." [CRD, 1998, pp.1-3] The 
CRD does not supercede any of the existing DoD acquisition directives (DoD 5000/8000 series), 
instead the intent is to provide a single source document that "sets the performance goals which 
DoD’s future MILSATCOM programs and commercial services should strive to achieve within 
available funding limits." [CRD, 1998, pp.1-3] 
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Figure 6. Required System Characteristics [CRD, 1998, pp.4-2] 
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As previously indicated and as shown in Figure 6, the CRD defines seven required 
system characteristics. The CRD does a very good job developing these characteristics and 
relating them to DoD’s military operational requirements. The approach taken by the CRD is 
similar to that taken in this thesis. The CRD constructs a DoD profile whose characteristics are 
derived from JV2010 and reflect the unique and critical warfighting capabilities of current and 
future U.S. forces. These capabilities are then associated with seven SATCOM terms, labeled 
required system characteristics. Defined in terms of user requirements each of the seven 
requirements directly relates a collection of user needs to SATCOM (spacecraft, constellation and 
terminal) design characteristics. The next section, 2. The Seven Characteristics, describes each 
in detail. 

The value of the CRD is summarized below; 

• It is an over-arching operational requirements oriented document that defines the 
minimum performance requirements of the future MILSATCOM architecture. 

• It is JROC validated and therefore is a valid and recognized source document for 
MILSATCOM programmatic decisions. 

• The document provides jointly agreed upon (SWarF reviewed and JROC validated), 
required system characteristics for MILSATCOM systems. 

2. The Seven Characteristics 

The CRD presents the required characteristics as joint, DoD wide "capstone" 
requirements that apply to all types of SATCOM, (refer to Table 1). [CRD, 1998, pp.1-18] Each 
characteristic is connected to DoD-defined mission areas and is intended to specify how mission 
areas are supported by MILSATCOM. Furthermore, five characteristics have some attributes that 
are "so critical to warfighters and their support communities they are designated as capstone 
KPPs," (key performance parameters). "By definition, KPPs are those capabilities or 
characteristics considered most essential for successful mission accomplishment." [CRD, 1998, 
pp.1-19] The KPPs are presented in Table 2. The CRD presents KPPs as system-of-system 
threshold requirements. Use of KPPs as threshold requirements does not require every future 
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component system of the MILSATCOM architecture to meet or exceed very KPP level of 
performance, instead KPPs define the performance level of the entire architecture. Therefore: 



• The seven required characteristics establish the link between the user needs and 
SATCOM system attributes. 

• KPPs provide specific guidance for threshold level performance of the entire architecture. 

• It is understood that each component system will contribute uniquely to the overall 
MILSATCOM architecture and that and trades will be conducted to optimize each 
component towards obtaining a threshold capable system-of-systems. 

• KPPs are JROC validated. 
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CATEGORY 


RATIONALE (WHY REQUIRED) 


COVERAGE 


Global national interests and threat environment. 

Regional conflicts/crises unpredictable in location, time, intensity and duration. 
Smaller US force structure; globally dispersed land, sea, air, and space 
operations. 

Time and geographically varying user population densities. 


CAPACITY 


Warfighter information demands are growing in response to doctrine and 
technology. 

Information + C4ISR + Precision Munitions = Combat Power 
Connectivity cannot be a limiting factor in the application of combat power. 
MILSATCOM = Assured warfighter connectivity when/where needed. 
MILSATCOM provides dynamic, multiple information transfer capabilities. 


PROTECTION 


Our C4I is a prime target and a center of gravity which we expect adversaries to 
attack. 

Must deny adversaries the ability to decapitate our C2 and 1ST capabilities. 
Nuclear deterrence remains a top DoD priority (and requires survivable C2). 

Must provide anti-jam, protection from SIGINT, information security, and other 
defensive information operations measures. 


ACCESS AND 
CONTROL 


Access to Information and comm on-demand: fundamental need of the warfighter. 
Warfighters must have control over their information and MILSATCOM domains. 
Military resources must be rapidly and dynamically reconfigured to respond to 
changing operational situations and priorities. 


INTER- 

OPERABILITY 


MILSATCOM is the space portion of the Defense Information Infrastructure. 

Most operations are joint in nature and execution (Land, Air, Naval, Mobility, 
Combat Support and Special Operations Forces). 

US Forces conduct missions with Allied, Coalition Partners, and Government 
Agencies. 

Warfighters use a variety of communications to effect needed Information 
transfers. 


FLEXIBILITY 


Warfighters prosecute military operations across a wide spectrum of conflict. 

Need to accommodate evolving doctrine, requirements, threats, and technologies. 
Emphasis is on fast-paced mobile operations. 

A wide variety of operating frequencies is required to support the warfighters’ 
needs. 

Warfighters’ must make efficient use of limited frequency spectrum. 

Systems should be reliable, easy to use and safe to operate. 


OUALITY OF 
SERVICE 


Supported warfighting and combat support systems drive performance criteria. 
Information must be transferred accurately and unambiguously. 



Table 1. Required System Characteristics [CRD, 1998, pp. 1-19] 
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PARAMETER 


THRESHOLD(KPPs) 


OBJECTIVE 


COVERAGE 


Ability to provide MILSATCOM when/where 
needed in areas north of 65 degrees south 
latitude 


Ability to provide MILSATCOM 
at all latitudes and longitudes 


CAPACITY 


Provide requisite amounts of wideband and 
narrowband capabilities (throughputs and 
accesses) to the warfighters and their 
supporting infrastructures: 

-Wideband (symmetric, asymmetric and 
broadcast) 

>Focus on deployed forces and OCONUS 
warrior support activities (e.g. DISN, 
Diplomatic Telecommunications Service 
[DTS], the intelligence community, etc) 
-Protected communications (see below) 
-Narrowband (netted and other topologies) 
>Sustain UFO capabilities and augment with 
MSS/PCS 


Threshold plus support 
projected growth rates. 
Support data rates up to 64 
KBPS into handheld 
narrowband devices. 


PROTECTION 


Provide levels of protection to sub-sets of the 
overall MILSATCOM capacities: 

-Survivable and anti-jam communications for 
NCA/SlOP forces 

-Anti-jam for "front line" C2 and common-user 
networks 

LPI/LPD for critical tactical and strategic 

covert/sensitive users 

"US Control for selected users (e.g. vital 

diplomatic and intelligence needs and selected 

tactical) 

Prevent unauthorized access to, or disclosure 
of, information. 


Threshold plus provide 
AJ,LPI/LPD and/or US Control 
for lower priority tactical, 
strategic & supporting 
networks. Automatically 
detect, characterize, & 
neutralize offensive 
information operations. 


ACCESS AND 
CONTROL 


CINCs/Joint Task Forces dictate resource 
utilization over apportioned resources and can 
plan, allocate, and schedule access within 
fractions of hours to a few hours. 
MILSATCOM resources can be rapidly and 
dynamically configured and re-configured 
within a few hours to fractions of hours 
(selected networks within minutes). 


Near-real-time authorization, 
denial, preemption of access. 
Accomplish dynamic resource 
configuration within a few 
minutes. 


INTER- 

OPERABILTY 


Interoperability between/among CINC and JTF 
components (e.g. Land, Air, Naval, Mobility, 
Combat Support, and Special Operations 
Forces) 

MILSATCOM is fully integrated as the space 


Threshold plus interoperability 
with allies and coalition 
partners and other Federal 
agencies (non-DOD). 



Table 2. Capston Key Performance Parameters [CRD, 1998, pp. 1-20] 
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The two previous tables provide the top-level connection between the required system 
characteristics, the KPPs and user needs. Being a joint document that addresses all types of 
SATCOM service, there are a subset of elements presented in the tables that are directly 
applicable to the naval wideband discussion. At the very least the required characteristics and the 
associated KPPs provide a legitimate baseline for the discovery and refinement of naval specific 
performance measures. The intent of the remaining portions of this chapter is to use the CRD 
required characteristics and KPPs to derive specific performance measures that articulate naval 
wideband SATCOM needs. 



D. SEVEN FUNCTIONAL OBJECTIVES FOR NAVAL WIDEBAND SATCOM 

The process of defining naval wideband SATCOM performance parameters is performed 
using the CRD's required systems characteristics as a starting point. The CRD is straightforward 
and intentional in its description of how the required characteristics and KPPs should be used. 
The CRD states. 



"The required system characteristics and associated thresholds and objectives 
identified in this section are presented at a system-of-systems ’capstone’ level. 
They provide the capstone requirements framework and performance goals to 
support the preparation of ORDs and other acquisition and leasing programs and 
initiatives. They also provide a mechanism for the JROC to maintain oversight of 
the mission area. The ORD remains the defining acquisition document for 
individual system development and testing. When individual system ORDs are 
developed they will contain detailed and specific KPPs and critical system 
characteristics, along with specific and qualified objective and threshold values. 
For their individual programs, supporting the obtainment, in whole or part, of the 
capstone requirement contained herein. Additional required system 
characteristics may be added, and some of the required system characteristics 
identified here may be modified or combined to better describe specific system 
requirements." [CRD,1998, pp.4-2] 



The use of the required characteristics as the functional objectives of a system evaluation 
hierarchy is in line with the intended use of the CRD characteristics. What needs to be done is 
that the joint, top-level characteristics need to be refined and tailored to more closely match naval 
wideband SATCOM characteristics. The translation of each of the seven required characteristics 
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to naval wideband user functional objectives is begun in the next section and parallels the 
development of the seven characteristics as presented in Section 4, Required System 
Characteristics, of the CRD. 

E. COVERAGE 

1 . Defined 

Coverage refers to the "part of the Earth that a spacecraft instrument^ can see at one 
instant or over an extended period" of time and is frequently a key element in spacecraft design. 
[Larson, 1992, pp.161] This definition implies that coverage has two attributes, time and 
geographic (earth surface area). A spacecraft’s antenna technology, the number of satellites in a 
constellation, and the constellation’s orbital parameters are critically linked to both of these 
attributes. Coverage is often the driving reason for selection of a space based system over a 
terrestrial system and therefore significant design, cost and performance trades are made to 
characterize a system’s time and geographic coverage . 

The CRD defines coverage as "the portion of the earth’s surface and the airspace above 
it which SATCOM services can be provided." [CRD, 1998, pp. 4-3] This definition fails to account 
for the time attribute, and therefore falls short of full characterization of coverage. Despite this 
shortcoming the CRD does include the time element of coverage in the discussion of coverage 
requirements and the refinement of the coverage KPP. Ultimately the CRD states that “the 
essential element of coverage is the ability to dynamically focus required satellite caabilties where 
the varieties of users are located across the face of the globe when they need it." [CRD, 1998, pp. 
4-10] Clearly this definition of coverage is in line with a space system designer’s definition of the 
term. Therefore, the definition of coverage that will be used in this COMMER wideband evaluation 
hierarchy is (refer to Figure 7. Coverage Defined): 



^ Instrument in this context refers to the spacecraft’s wideband communications antenna. 
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COVERAGE 



The ability to dynamically focus required satellite 
capabilities when and where needed. 



Figure 7. Coverage Defined 
2. Why is it Required? 

Given a definition for coverage the next step in the process is to identify what user 
characteristics coverage encompasses. The CRD’s capstone level justification of coverage must 
be translated into naval warfighter characteristics, and coverage must support those 
characteristics. This translation is displayed in Table 3. Coverage Justification. 



CATEGORY 



Coverage • 






RATIONAL (WHY REQUIRED) 
DoD/JOINT REQUIREMENTS 

Global national interests and threat • 
environment. 

Regional conflicts/crises 
unpredictable in location, time, • 

intensity and duration. 

Smaller US force structure; 
globally dispersed land, sea, air, • 

and space operations. 

Time and geographically varying 
user population densities. 






RATIONAL (WHY REQUIRED) 
NAVAL REQUIREMENTS 

Expeditionary nature implies 
unpredictable in location (global), 
time, intensity and duration. 
Structured to project power implies 
coverage of unpopulated areas (the 
sea). 

Self-supporting implies coverage to 
a smaller naval force structure; 
globally dispersed land, sea, and air 
operations. 

Unrestricted coverage not limited by 
accepted international treaty 
Scalable implies the ability to 
increase/decrease coverage to 
meet operational need. 



Table 3. Coverage Justification 

To support the naval warfighter acceptable alternatives should not negate the naval 
force’s expeditionary and self-supporting nature. Ideally coverage must be available to all units, at 
all locations with out the luxury of a priori location information. Furthermore, ships at sea (an 
unpopulated area without a strong commercial market) must be able to participate in the naval 
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network. Coverage must not be limited by international regulations that impose frequency 
restrictions on ships operating in littoral areas.^ Lastly, coverage must be scalable, capable of 
supporting peacetime operations up to a major theater of war (MTW). 

3. Measuring Coverage, Where is Coverage Required? 

In this evaluation hierarchy the coverage attributes time and geographic are used to 
quantify a system’s coverage. This section will address the geographic space performance 
measures and Section 4. Measuring Coverage, When is coverage required? will address the time 
driven measures. 

In geographic terms coverage is defined as, the discrete portion of the earth’s surface that 
receives a usable amount of a communications system’s power. Two elements of this definition 
need refinement. They are, 1) the discrete portion of the earth's surface, and 2) usable amount of 
power. In order to effectively refine the definitions of these elements two additional SATCOM 
designer terms must be introduced, both of which contribute to the understanding of geographic 
coverage. 

These terms are: 

Footprint Area {FA), also Field of View (FA) or instantaneous coverage area) = 
the area that a spacecraft’s antenna can see at any instant . 

Instantaneous Access Area (lAA) = all the area that the spacecraft’s antenna 
could potentially see at any instant if it were scanned through its normal range of 
orientations. [Larson, 1992, pp.161-162] 

As displayed in Figure 8, FA and lAA are clearly not equal surface areas. The lAAs of the 
Global Broadcast Service (GBS) payloads hosted on the geosynchronous UHF Follow-On (UFO) 
satellites 8, 9 and 10 are the discrete portions of the earth that any one of the three steerable 



^Naval forces operating within a nation's territorial waters are subject to the radio spectrum usage 
regulations that nation has established. These regulations can significantly restrict use of mobile 
SATCOM if it interferes with terrestrial communications. 
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downlinks (1 - 2000nm wide area, and 2 - 500nm spot beams) can be scanned through. ^ 
Therefore, as shown in Figure 8, the black ovals correspond to the GBS lAAs, which are 
approximately 150-degrees wide and encompass all latitudes between 65-degrees north and 65- 
degrees south. Furthermore, although the name insinuates "instantaneous" scanning of the lAA, 
this is not always the case. For example, for GBS antenna pointing (relocation of a spot beam) 
can take up to ten minutes to complete. [G6S JCO, 1997, pp.14] 




Figure 8. Example of FA and lAA [USPACOM GBS CONORS, 1998, pp. 11] 

Also represented in Figure 8, is the GBS FA. Since each GBS payload has three 
steerable antennas there are three distinct FAs per payload. In the case of GBS there are two 

^ Although GBS is not a two-way broadband system (users are receive only), GBS can still be 
used in this example for explaining lAA and FA. 
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500nm spot beams (FAs) and one 2000nm beam (FAs). Users who are within a particular GBS 
lAA but not within a GBS FA will not receive the transmitted broadcast . The dramatic and 
operationally significant difference between lAA and FA should now be clearly evident. 

There remains a final point to fully define a satellite’s lAA and FOV and that is; what is the 
power level at the edge of the FA when it is located at the edge of the lAA? The goal here is have 
a common standard for vendors to use when depicting lAA and FA. The result is the ability to 
compare competing systems on equal grounds. The answer to the power level question is given 
by the definition of the second element of the geographic coverage definition. 

The second element of the geographic definition, usable power, is a multi-parameter 
variable that is highly application dependent. A SATCOM terminal’s usable power is a function of: 
user population, terminal type, terminal size, operating frequency, weather, elevation angle, 
desired data rate, desired bit error rate, and numerous other parameters that degrade signal 
energy. Simply put, usable power varies with the situation. Despite this variability the user can 
create a common, effective definition for usable power. An example definition of usable power is: 
The power needed to provide a standard terminal a bit error rate (BER) of 10'^° or better while: 

• (Location) operating at the edge of the FA when it is located at the edge of the lAA. 

• (Environmental conditions) in clear weather and stable sea conditions [Sea State 1] 

• (Elevation angle) with an antenna elevation angle of greater than or equal to 10-degrees 

Obviously each component of this definition needs some clarification. First and foremost, 
in order to create a common point of reference for the comparison of different commercial 
wideband service providers the user must define a standard terminal. Characterization of a 
standard terminal should come directly from the market survey and should reflect the user’s 
needs. Because the characterization of a standard terminal is driven by user needs some degree 
of subjective will be evident. Weight and size are the two most critical terminal characteristics. 
Both the antenna and supporting hardware need to be included in the user’s definition of a 
standard terminal. In an effort not to unintentionally exclude service providers the user should 
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characterize the terminal weight and size using acceptable ranges, or "not to exceed values". 
Although antenna sizes are often presented in terms of dish diameter, antenna volume should be 
used instead. In the naval environment (aboard ship, in a periscope mast, or on the top of a 
vehicle), space for new equipment is often limited and severely constrained. Furthermore, harsh 
environmental factors (humidity, salt, sand, mud, etc.) generally require that all sensitive 
equipment be protected by some hood or casing. Environmental protection equipment can 
significantly increase terminal volume. Therefore, specification of terminal weight and size should 
include all terminal components. 

The determination of an acceptable BER is probably the most important element in the 
definition of usable power. Acceptable BER determines what products and applications the user’s 
terminal will support. As a critical component of the link budget equation, BER is a driving 
component in the design of communication systems. Acceptable BER should be entirely user 
determined and needs driven. Table 4 lists the acceptable BER for different applications (required 
data rate). This chapter’s later section discussing Quality of Service provides a greater 
explanation of BER. The use of a BER better than or equal to 10'^° was based on GBS Phase II 
requirements. [GBSJCO, 1997, pp.17] 



Application 


Acceptable BER 


Speech Telephony 


1x10'^ 


Telegraph/Telex 


1x10"' 


Computer Data 


- 1x10® 


Computer Networks 


1x10'^ 



Table 4. Standard BERs [Freeman, 1991, pp. 417] 



Location of the terminal is another critical factor, in that transmission path losses are 

^ 1 



inversely proportional to the square of the distance from the transmitter. 



loss a 



/-I 



distance^ 

Therefore, transmission losses are maximum when the edge of FA coincides with the edge of the 
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lAA. Point A in Figure 8 represents such a point.® The receive antenna elevation angle is also 
related to transmission path loss. The lower the angle the greater the distance between the 
transmitter and receiver and therefore the greater the transmission loss. Restricting elevation 
angle to greater than or equal to 10-degrees establishes a common boundary condition that 
reduces the parameters involved in determining available power. For reference, in order to block 
the receiving antenna an obstruction (i.e., another ship) positioned 1.0 nautical miles from the 
receiver would have to be over 0.176 nautical miles tall (327m) (refer to Figure 9. Elevation 
Angle). In most cases only land or tall objects close to the receiver will obstruct its view. 

A 

tan (lOdeg) = 0.176 nm 



Figure 9. Elevation Angle 

Specification of the operating conditions as clear weather and stable sea state [SSI] 
further reduces the variables involved in the power determination problem and creates a common 
point of reference for system comparison. Although the naval user does have requirements to 
operate the terminal at greater than SSI this does not need to be reflected in defining the 
requirements for FA and lAA. Recall that the goal is to create a common reference point for 
comparing competing system capabilities. Therefore, use of ideal weather and sea conditions 
removes ambiguities that vendor’s can introduce when trying to market their system. 




< > 

1 nautical mile 



® Note: Use of GBS as an example does not imply that GBS conforms to the definition of usable 
power presented above. 
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The bottom line for characterization of a standard terminal is to create a user needs 



driven definition that provides a common reference point for vendor comparisons. The desired 
output is to have vendor’s provide coverage maps {lAA and FA) that are based on identical 
parameters.^ Furthermore, the vendor and the user have a mutual understanding of how 
geographic coverage is defined. That is, geographic coverage is the discrete portion of the earth’s 
surface that receives a usable amount of a communications system’s power. Given that the user 
has clearly identified an acceptable BER for a standard (user defined) terminal. Armed with an 
understanding of what capability a vendor’s geographic coverage maps represent the user can 
discuss coverage in operational terms as geographical focus areas. 

a) How Does the User Describe “Where” He Needs Coverage? 

Three terms commonly used to characterize coverage are global coverage, 
geographic coverage, and worldwide coverage.” DoD defined a fourth term, theater coverage, to 
articulate the need to “provide for concentrations of in-theater forces." [CRD, 1998, pp.4-12] 

Global coverage is defined as all latitudes and longitudes and geographic 
regions. "[Cf?D, 1998, pp.4-3] The need for global wideband coverage is directly tied to the naval 
force global presence mission and expeditionary nature. Geographic coverage refers to any one 
of six geographic regions (refer to Figure 10): 

• CONUS and Americas 

• Atlantic-Europe-Africa 

• Asian-Indian Ocean 

• Pacific 

• North Polar (all longitudes above 65-degrees north latitude) 

^ Most coverage maps are based on "common" regulatory requirements and reflect a system's 
lAA and FA as determined by frequency interference guidelines. Although important, these 
coverage maps fail to adequately depict a system from a user's perspective. The common 
reference point proposed in this thesis is user derived, and therefore is more directly meaningful 
to the user and more valuable when comparing competing systems. The bottom line the coverage 
map needs to directly represent where a user will get his desired level of service. 
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South Polar (all longitudes below 65-degrees south latitude) [CRD, 1998, pp. 4-3] 




NORTH POLAR 



SOUTH POLAR 



Figure 10. Geographical Regions (Representative) [CRD, 1998, pp. 4-3] 

The driving need behind regional coverage is the requirement to support widely 
dispersed naval forces operating in areas vital to the nation’s interests. Additionally inter-regional 
coverage is required to ensure that communications between CONUS and a given region is 
maintained. The “sixth region, the South Polar Region, has had very few MILSATCOM 
requirements identified in the past. However, that region may receive emphasis if DoD user 
requirements emerge in the future,” (see Figure 11). [CRD, 1998, pp. 4-3] 

On the other hand the North Polar Region (refer to Figure 12) has significant 
historical and future importance to naval force execution of the national security strategy. Although 
there are fewer requirements for North Polar coverage than any of the other four regions 
(excluding the South Polar Region) the North Polar needs are considered vital to the national 
interests and therefore receive high priority. More information concerning north polar 
communications requirements are provided in the classified Polar System ORD, (see List of 
References). [CRD, 1998, pp. 4-3] 
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Figure 11. South Polar Region [CRD, 1998, PP. 4-13] 
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“Worldwide coverage, is generally defined as the surface of the earth between 
65-degrees south latitude and 65-degrees north latitude.” [CRD, 1998, pp. 4-3] Therefore 
worldwide coverage encompasses the CONUS and Americas, Atlantic-Europe-Africa, Asian- 
Indian Ocean, and Pacific geographic regions. The poles are not part of worldwide coverage. The 
shaded region of Figure 13 represents worldwide coverage. Similar to the need for global 
coverage, worldwide coverage is needed to support “dispersed, distributed, mobile, and 
relocatable users... from anywhere... without the necessity of their time and location of access 
always being known beforehand.” [CRD, 1998, pp.4-11] This requirement clearly is in line with the 
naval force characteristics of expeditionary nature, structured to project power, self-supporting, 
unrestricted and scalable. 




Figure 13. Worldwide Coverage Area (65 Deg N • 65 Deg S) [CRD, 1998, pp. 4-11] 



The final geographical region to be defined is the Theater coverage region. As a 
joint level document the CRD presents the need for coverage in support of two near simultaneous 
major theaters of war (MTW) or multiple small-scale contingencies (SCC). [CRD 1998, pp. 4-12] 
As participants in a MTW naval forces share in the need for a SATCOM “surge” capability, to 
support the greater communications demand in a MTW. However as potentially the primary 
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participants in SSCs (i.e., an Amphibious Ready Group (ARG) with an embarked Marine 
Expeditionary Unit, Special Operations Capable (MEUSOC)), naval forces have a definite need for 
theater level coverage. Not only is there a requirement to provide coverage of a SSC theater but 
there is an additional requirement to have the ability to “adjust the theater coverage in response to 
the tactical situation.” [CRD 1998, pp. 4-12] Additionally, the providing system must be able to 
provide out of/into theater communications between in-theater forces and CONUS or overseas 
support forces. [CRD 1998, pp. 4-12] As Figure 14 shows a SSC is approximately 1000 by 1000 
km in size. 




Figure 14. Notional Theater Area [CRD, 1998, pp. 4-13] 

b) Geographic Measures of Performance (MOPs) 

The parameters defined in the previous sections; standard user terminal, usable 
power, acceptable BER, lAA, FA, global coverage, worldwide coverage, regional coverage, and 
theater coverage can be combined to form two MOPs that represent naval force geographic 
coverage needs. The two MOPs are lAA and FA. That is, a candidate system’s lAA and FA can 
be defined by the level of coverage (global, worldwide, regional, theater) it provides. Then the 
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geographic coverage of competing systems can be compared based on their lAAs and FAs. Since 
each system’s lAA and FA should be defined in accordance with the definition provided in this 
thesis, two systems claiming to have worldwide lAA and FA coverage should provide the same 
amount of usable power to a user defined standard terminal. It is important to note that in most 
cases lAA and FA coverage will not be equal. For example, as a system GBS Phase II (once 
completed) has a regional lAA,^ and a theater FA (taken together the three spot beams can 
adequately cover a theater). On the other hand Teledesic intends to use a large number of LEO 
spacecraft each having numerous spot beams to effectively provide worldwide lAA and FA 
coverage. Of course a single Teledesic satellite will not provide worldwide coverage, but the 
Teledesic customer won’t purchase service from a single satellite, they will purchase service from 
the entire system. Although this may seem obvious, it is only intended to highlight the need to 
evaluate competing systems at the overall service level not satellite level. With that said if a 
company wants to sell regional coverage using a single GEO satellite then the overall service 
level equals that single satellite’s capabilities. 

In summary geographic coverage is the discrete portion of the earth’s surface 
that receives a usable amount of a communications system’s power. The user determines usable 
power by defining the characteristics of a standard terminal (acceptable BER, location, 
environment, and elevation angle). The difference between a system’s lAA and FA is identified 
and articulated in terms of global, worldwide, regional and theater coverage. Figure 15 provides a 
graphical review of the terms presented. 



® Given the gap in coverage between UFO-8 and UFO-9, GBS Phase II is not a true worldwide 
system. The central U.S. is not in the GBS lAA and therefore major commands in that region 
(USSPACECOM, USTRANSCOM and USSTRATCOM) are not provided direct access to GBS 
Phase II this gap prohibits classification of GBS as having worldwide lAA. 
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MOE = COVERAGE 



The ability to dynamically focus 
required satellite capabilities 
when and where needed. 




GEOGRAPHIC COVERAGE 
The discrete portion of the 
earth’s surface that receives a 
usable amount of a system’s 
power. 
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Figure 15. Geographic Coverage Summary 
4. Measuring Coverage, When is Coverage Required? 

As previously stated coverage is the ability to dynamically focus required satellite 
capabilities when and where needed. As described in Chapter II, naval wideband SATCOM users 
will operate in a dispersed, highly dynamic environment, often requiring short notice relocation. 
Therefore, naval users require wideband satellite services that provide coverage when and where 
required “without the necessity of their time and location of access always being known 
beforehand. 1998, pp. 4-11] This does not necessarily imply that in order to be useful a 
system must have 24 hour a day geographic coverage. What it implies is that usefulness, (value 
to the warfighter) is the ability to rely on having coverage in a particular geographic area when it is 
needed. Within this evaluation hierarchy accounting for reliable access is performed in two places; 
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once under the coverage functional objective (this section) and again under the access and 
control functional objective (in a later section). Accounting for time under the coverage MOE 
provides a means for ensuring that access to the satellite is possible because the satellite is in 
view of the user’s terminal. Under the access and control MOE, it is assumed that the satellite is in 
view of the user’s terminal and that access is a function of transponder loading and user priority. 
The difference is subtle but important. Furthermore, being in view of the satellite implies that the 
user’s terminal is within a system’s FA and therefore can receive service from the system. 
Therefore, time coverage answers the question; How long and how often does the user have 
access to the FA7 

The parameters that determine whether or not a satellite is in view of the user’s terminal 
are functions of the design of the overall SATCOM system. The primary drivers of "time in view" 
are satellite altitude, inclination, and the number and spacing of the system spacecraft. A lengthy 
discussion of orbital characteristics is not required, however it is important to have an 
understanding that in general, (refer to Figure 16): 

• Geostationary & Geosynchronous (GEO): satellites positioned above the equator at an 
altitude of 19,323 nautical miles (35786 km) can generally be said to remain in a fixed 
position relative to a position on the surface of the earth. Furthermore due to their 
increased altitude GEO lAA and FA are often very large. 

• Medium Earth Orbiting (MEO): satellites at altitudes between LEO and GEO move relative 
to a fixed position on the surface of the earth. Time in view, lAA and FA for MEO satellites 
varies and is greatly determined by the oblateness (eccentricity) and altitude of the 
satellite’s orbit. 

• Low Earth Orbiting (LEO): satellites generally at altitudes of less than 540 nautical miles 
(1000 km) move relative to a fixed position on the surface of the earth, and the average 
time in view for the generic LEO spacecraft is less than 17 minutes. Furthermore, given 
that the earth rotates underneath the moving satellite the time between opportunities to 
view the satellite varies from a low of approximately 90 minutes up to a high of many 
hours. In general LEO lAA and FA are much smaller than that of a GEO satellite. 



These short definitions reveal the critical impacts that orbit selection 
warfighter. For example, it should be apparent that LEO and MEO systems will 



has on the 
require user 
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terminals to have a satellite tracking capability.^ That is, the user’s antenna will have to acquire 
and then track the moving satellite. Although this problem is not technologically infeasible it does 
increase the complexity of the system. However, a benefit of the LEO systems is that their relative 
close proximity to the user generally allows the user to have a smaller less powerful terminal 
compared to that needed for a MEO or GEO system. Less power translates into less complexity, 
less logistics support, and decreased electronic signature. The underlining fact is that numerous 
trades go into SATCOM coverage optimization. The naval user should remember that commercial 
wideband systems will be optimized for the commercial (not naval) user, and therefore the Navy 
and Marine Corps will have limited influence over system design. 




Figure 16. Representative Satellite Coverage Patterns [CRD, 1998, pp. 4-5] 



This brief introduction to the orbital operating regimes can be used to understand that 
time coverage (access to the satellite) must be measured by two parameters: 



® A characteristic of the SHF frequency bands (the band in which most of the wideband systems 
operate) is that their transmissions are highly focused and directional. This implies the need for 
satellite tracking. 
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Time in view {TIV): where 7/1/ is the maximum continuous period of time that the 
user with a standard terminal is within the system level FA, (i.e., the time that the 
user is able to communicate with the system.) 

Time between views (TBV): where TBV is the maximum continuous period of 
time that the user with a standard terminal is not within the system level FA, (i.e., 
the time that the user has to wait to use the system.) 

For the user the goal is to maximize TIV while minimizing TBV. For example Astrolink 
(shown in Figure 17), a proposed wideband system supported by Lockheed Martin will employ 
nine geostationary satellites in five orbital slots to ensure 24 hour worldwide coverage. Therefore, 
since GEO satellites do not move relative to a position on the earth, system level TIV equals 24 
hours, 761/ equals 0 hours^*^, an ideal system in terms of time coverage. 

Contrast the Astrolink solution with a generic LEO system that consists of only one 
satellite. Given that a LEO satellite moves quickly relative to a fixed position on the earth TIV is 
approximately 17 minutes or less and TBV ranges from approximately 90 minutes to hours 
depending on the orbital inclination and altitude. In order to get a system level TIV equal to 24 
hours, and 761/ equal to 0 hours the LEO constellation must consist of a large number of properly 
phased satellites. As currently planned, Teledesic, a proposed commercial wideband system, 
intends on having 288 satellites in LEO^^ to ensure TIV equal to 24 hours, and TBV equal to 0 
hours. ^^[Teledesic, 1998] Again for the user this represents ideal time coverage. 



This is assuming that Astrolink will provide both a worldwide lAA and FA. 

^ ^ Teledesic is filed to operate at an altitude under 1400km, which is still considered LEO. 
[Teledesic, 1998] 

Just as in the Astrolink case it is assumed that Teledesic will provide an effective worldwide 
system level lAA and FA. Teledesic intends to do this by employing multiple overlapping spot 
beams on each of its 288 satellites. [Teledesic, 1998] 
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Figure 17. Astrolink Satellite Constellation [Astrolink, 1998] 

5. Summary 

In summary time coverage uses the measures of time in view (TIV) and time between 
views (TBV) to answer the question how long and how often the user has access to the FA7 
Again, the goal is to provide the user with maximum TIV and minimum TBV. In conclusion as a 
functional objective the coverage MOE consists of four MOPs: FA, lAA, TIV and TBV. Relative 
weighting and determination of dropdead, threshold and objective values for each of these 
measures are provided in Chapter IV. Figure 18 and Figure 19 provide a graphical review of the 
terms presented in this section. 
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Figure 18. Coverage Measures of Performance 
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Figure 19. Coverage Summary 



F. 



CAPACITY 



1. Defined 

Most communications engineering textbooks define capacity as the "maximum rate of 
reliable information transmission over a given channel." [Haykin, 1983] Prior to the late 1940’s the 
leading information transmission theorists believed that the "output of a noisy channel could only 
be imperfect data."[Stark, 1988] It was known that output errors could be decreased by increasing 
power, channel bandwidth or both, “but perfect reliability at a fixed power and bandwidth was 
thought to be impossible in principle."[Stark, 1988] In 1948, in his bandwidth paper, Claude 
Shannon successfully argued that "virtually perfect transmission was possible in a given channel 
at any rate less than a fixed C bits/channel," where C was the channel capacity. [Stark, 1988] The 
essence of Shannon’s theory is that due to errors in the channel (i.e., noise in the transmission 
path) the output information (symbol) may be different from the input information (symbol) and 
there exists a theoretical value below which channel induced errors are probabilistically small. 
Furthermore, this theoretical capacity is achieved by proper coding of the input signal. Numerous 
complex yet effective coding schemes have been derived from this fundamental theorem. Each 
coding scheme has distinct benefits and detractors that impact bandwidth, power and BER. 

Coding theory and the engineering that goes into channel capacity optimization is not 
required knowledge for naval warfighters trying to articulate their SATCOM capacity needs. 
However, the basic principle that capacity comes at a price and that achieving desired capacity 
involves trades that can impact mission effectiveness is essential knowledge. From this it should 
also be realized that capacity is inherently a function of overall system design. Although follow-on 
modification of coding schemes or information transmission protocols may increase a system’s 
capacity satellite hardware components and constellation design will pre-determine the 
boundaries of these “soft" modifications. The naval warfighter’s goal is to associate capacity 
requirements with SATCOM design parameters. Doing so will indicate preference for particular 
system capabilities and therefore design options. 
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The optimization of SATCOM capacity not only includes determination of frequency 
(bandwidth), power, and coding scheme but numerous other variables. The CRD does a good job 
of presenting throughput (capacity) as a function of some of the variables, (see Figure 20). 

THROUGHPUT versus 
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Figure 20. Throughput Parameters [CRD, 1998, pp. 4-3] 



Given these parameters the CRD defines capacity of a SATCOM system as "the kinds 
and amounts of throughput available to the targeted user population as measure against specific 
employment scenarios. Capacity also encompasses the numbers of individual ’accesses’ that can 
be supported by the system." [CRD, 1998, pp. 4-3] This definition does a good job limiting 
capacity to a specific situation (i.e., employment scenarios and number of accesses). 




Figure 21. Example User Network Topologies [CRD, 1998, pp. 4-5] 
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The number of accesses is used to account for the number of individual demands that are placed 
on the transmitting satellite. Access varies with network topology and is critical to determining 
overall system loading. As presented in the CRD Figure 21 shows how the number of accesses 
varies with network topology. For example, the hub-spoke topology has four accesses, H, 1,2, 
and 3. [CRD, 1998, pp. 4-4] It is important to realize that a SATCOM broadcast is one access, 
because it only places one demand on the satellite transponder. However, when an individual 
receiving element attempts to reach-back to the hub an additional access is created. Furthermore, 
each access does not have to require the same capacity. 

The complexity of characterizing a user’s capacity should now be evident. Required 
capacity depends on the situation. However, in an attempt to define capacity such that it can be 
used to evaluate competing wideband concepts for naval use across a wide spectrum of 
operations and situations is important to include this flexibility in the definition. This flexibility is 
represented in the definition displayed in Figure 22. What this definition means and what 
performance measures are used to represent is the focus the section titled, 3. Measuring 
Capacity. 



CAPACITY 

The maximum rate of reliable information 
transmission. 



Figure 22. Capacity Defined 
2. Why is it Required? 

Given a definition for capacity the next step in the process is to identify what user 
characteristics are addressed by the capacity MOE. In general capacity requirements are driven 
by an increase in the warfighter’s need for information. New weapon technologies (i.e., precision 
guided munitions), new reconnaissance systems (e.g., unmanned aerial vehicles) and emerging 
surveillance and targeting systems such as the Cooperative Engagement Concept (CEC) require 
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wideband communications for effective employment. Wideband SATCOM, possibly commercial 
wideband SATCOM will be used to met these needs. In an ideal situation naval forces at every 
level would have enough wideband capacity to satisfy user requirements. Translation of the CRD 
joint level capacity requirements to naval force requirements is shown in Table 
5. 



CATEGORY 



Capacity • 






RATIONAL (WHY REQUIRED) 
DoD/JOINT REQUIREMENTS 

Warfighter information demands 
are growing in response to 
doctrine and technology. 
Information + C4ISR + Precision 
Munitions = Combat Power 
Connectivity cannot be a limiting 
factor in the application of combat 
power. 

MILSATCOM = Assured warfighter 
connectivity when/where needed. 
MILSATCOM provides dynamic, 
multiple information transfer 
capabilities. 






RATIONAL (WHY REQUIRED) 
NAVAL REQUIREMENTS 

Expeditionary nature implies a need 
for capacity on the move (when and 
where needed). 

Structured to project power implies 
adequate capacity to support 
advanced weapon systems. 
Capacity can not limit combat 
power. 

Self-supporting implies a capability 
to provide user required level of 
capacity. 

Unrestricted implies assured 
access to required capacity. 
Scalable implies the ability to surge 
to meet increased demand. 



Table 5. Capacity Justification 

As with the other MOEs the lack of capacity should not hinder naval forces’ operational 
concepts. Capacity should advance and contribute to the Navy and Marine Corps’ expeditionary 
nature and enhance the naval forces’ ability to project power ashore. 

3. Measuring Capacity 

As previously presented, capacity is defined as a system’s maximum rate of reliable 
information transmission. Critical to this definition is the determination of reliable information. For 
use within this evaluation hierarchy reliable information equates to the BER as stated by the user 
in the definition of a standard terminal. Recall that a standard terminal was defined as a terminal 
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with a user specified; weight, volume, BER, location, operating environment and minimum 
operational elevation angle. This same standard terminal will be used as a constant parameter for 
all of the capacity MOPs. 

It is proposed that a system’s capacity be measured by three performance parameters: 
terminal mobility, quantity of accesses, and terminal data rate. Each of three MOPs has sub- 
attributes that are used to compute a numerical value to represent each MOP. In effect these sub- 
attributes account for a 4'^ level in the evaluation hierarchy and are MOPs to the MOPs. However, 
since this level is not present across all MOEs it was not originally presented. In an effort to 
reduce confusion, capacity will remain labeled a MOE; mobility, quantity and data rate are 
capacity MOPs; and the characteristics that define the individual MOPs will be referred to as sub- 
attributes. Discussion of how these sub-attributes are integrated to create a MOP value is 
presented in Chapter IV, Section E, Capacity MOPs. 

a) Data Rate 

A basic desire and fundamental design parameter is a system’s maximum 
throughput to a single operator. This is best measured assuming ideal conditions. Use of ideal 
conditions removes external variables that can bias vendor supplied results. In this case ideal 
conditions are those conditions specified by the user defined standard terminal. The goal of this 
performance measure is to allow the vendor to tell the user what the system is capable of 
providing to a single standard terminal under ideal conditions. For the commercial vendor this may 
be equivalent to the data rate that is advertised to the civilian customer. The naval user’s 
specification of a standard terminal is analogous to the fine print in the advertisement that defines 
the boundaries of vendor’s claims. The data rate MOP has two sub-attributes; terminal transmit 
data rate and terminal receive data rate. These two data rates are separated to account for the 
user’s tendency to conduct asymmetric communications. 
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b) Mobility 

User mobility is a critical naval user parameter. In terms of SATCOM design 
specifications user mobility has a significant impact on terminal design. The 21®* century naval 
warfighter will need to have wideband capacity on the move. Just how much capacity is required 
is a point of debate. It is the author’s opinion that the warfighter desires the same amount of 
information on the move as is required at the halt. The warfighter does not want to stop physical 
motion to get information (i.e., communications on the "pause"). However, given a choice naval 
warfighters favor mobility to capacity. Although speed of information exchange is as an element of 
warfare (i.e., information operations) and therefore critical to survival in combat, a platform’s (i.e., 
ship, plane, or vehicle) ability to physically move has a more direct relationship to survival. The 
bottom line is that a moving target is harder to hit; no matter how fast it processes information. 
With these thoughts in mind evaluation of a system’s ability to deliver a fixed amount of capacity 
to a mobile user is an important MOP. 

Just how much capacity is available to the mobile user is driven by how a mobile 
user is defined. For use within this evaluation hierarchy a mobile user will be characterized by 
four sub-attributes; velocity, pitch rate, roll rate and yaw rate. Each parameter is evaluated in 
isolation. Therefore one of the four parameters is varied while the remaining three are held 
constant. Capacity is also held constant and use of a standard terminal is assumed. The 
parameters are assumed to be measured using a standard terminal mounted on any platform 
capable of 3-axis motion (i.e., ship, plane, submarine and vehicle). The goal is to identify 
maximum velocity, pitch, roll and yaw rates for a fixed amount of capacity. It is assumed that the 
terminal remains in the FA throughout the period of motion. Therefore, the sub-attributes are 
actually measuring the antenna’s tracking ability. The fixed capacity value that is used in this 
evaluation should be user determined and constant across all evaluated systems. Therefore, for 



International regulations concerning FSS to a mobile user are not accounted for in this 
performance parameter and are assumed to not effect mobile SATCOM capacity. 
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the capacity mobility MOP the user must specify a capacity value in addition to characterization of 
a standard terminal. 

A user’s velocity is consider to be the rate of change in linear motion (i.e., pitch 

rate = roll rate = yaw rate = 0) on a ideal (flat) surface. The goal is to identify a terminal’s 

maximum attainable speed along a flat surface (i.e., a truck on a paved highway) for a fixed 
capacity. An example use would be an in-flight video collaborative planning session between 
assault forces embarked on multiple C-17 & C-141 enroute to a combat area. The velocity sub- 
attribute can be measured in either; knots, miles per hour, meters per second, or kilometers per 
hour. 

Pitch, roll and yaw rates are also intended to measure a terminal’s ability to 
provide a fixed level of service while executing tactical maneuvers or operating in harsh 

environments. Each parameter is measured in degrees per second (or radians per second) while 
the other three sub-attributes are held constant (velocity, and the other two axis motions). Pitch, 
roll and yaw are defined as (refer to Figure 23): 

• Roll: is motion about the X-axis. 

• Pitch: is motion about the Y-axis. 

• Yaw: is motion about the Z-axis. 




Figure 23. Mobility Attributes 
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Taken together the four mobility sub-attributes capture a system’s ability to 
provide wideband service to a mobile user. Antenna steering wold most likely be accomplished 
with a combination of phased array (electronic beam shaping) and mechanical tracking. 

c) Quantity 

Quantity of accesses is a desired parameter in that it provides a measure of 
overall system capacity. Naval vision publications and future naval concept recognize that the 
power that information has on the battlefield is related to the number of participants in the 
information network. Network participants not only receive information but they inject information 
and data into the network. The often cited Metcalfe’s Law, "the power value of a network 
increases as the square of the number of nodes on the network (N^)," (Robert Metcalfe, inventor 
of the Ethernet) is used to express the contribution that increased participants provide. [Boyd, 
1997] Therefore two sub-attributes are used to measure quantity of access. These sub-attributes 
are, 1) total accesses available in a theater^ and 2) available excess capacity (in terms of 
accesses) within a theater. Each of these two attributes are defined and justified below. 

Naval users will not be the sole users of a commercial wideband SATCOM 
system, and in a worldwide crisis demands for more SATCOM accesses will be made by the 
commercial customer. The Navy and Marine Corps will directly compete with other interested 
parties (i.e., television and news organizations, relief organizations, other allied military forces, and 
possibly enemy forces) for commercial wideband resources within the theater of operations. This 
fact is one of the major detractors for military use of commercial satellite service, wideband or 
otherwise. Assured access in terms of contractual guarantees during times of congestion is 
addressed in the Chapter III section titled Access and Control. 

The fofa/ sub-attribute is used to identify the maximum number of accesses that a 
vendor can provide to a theater. Accesses for a theater (MTW) vice any of the other coverage 

Theater as defined in the coverage MOE section of this thesis. Where a theater is a MTW and 
defined as an area approximately 1000km x 1000km. 



59 



regions is intentional. In terms of communications capacity management, concentrated need for 
SATCOM service is often a more difficult problem to handle than dispersed need. This is a 
common characteristic for any supply and demand driven service provider. The theater is the 
smallest geographical region and has the potential to have the highest concentration of military 
users. Therefore, it makes sense to identify a provider’s ability to accommodate large numbers of 
user in a fairly small region. 

A vendor’s capability to quickly respond to increased theater demand is 
measured using the excess sub-attribute. The goal of this sub-attribute is to validate a vendor’s 
stated capability to surge to meet demand, independent of any contractual agreements with users. 
This sub-attribute does not attempt to measure the vendor’s ability to dynamically increase theater 
capacity by launching additional satellites or activating on-orbit failure-mode spares as peak 
capacity spares. Instead the sub-attribute is intended to answer the question, how much excess 
capacity does the vendor have (on average) for a specified theater? 

The naval user’s desire for surge capacity is in direct conflict with a commercial 
vendor’s desire to operate a cost effective full time at or near capacity system. Among other things 
the commercial vendor associates profit with cost per unit of on-orbit capacity. A system that 
operates below capacity is under utilized and reduces profit. For the commercial vendor there is 
little difference between "war time" and "peacetime" each presents its own opportunities and 
market environment. The commercial vendor (other than a minor operating margin) has little 
incentive to maintain a robust surge capacity (i.e., hold capacity in reserve) in order to meet naval 
wartime requirements. Therefore, it is evident that commercial surge capacity will come at a price 
(monetary cost) to the naval community. It is the author’s opinion that within DoD there is little 
willingness to fund a "surge incentive," that would pay for reserve capacity. This does not imply 
that other solutions to the problem do not exist, it is only intended to highlight the competing 
interest of the naval forces and commercial SATCOM providers in terms of surge capability. 

An option would be to purchase more capacity than is required for peacetime operations and 
allow the service provider the option to resell unused capacity. DoD would receive a rebate for re- 
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Within the proposed evaluation hierarchy, excess capacity is reflected as a 
percent of the total capacity after naval peacetime access requirements and commercial access 
requirements within a given theater of interest have been taken into account, (see Figure 24). 
Evaluators will have to provide the commercial vendor with a chosen theater of operations, and 
rely on the vendor to supply an expected total commercial customer capacity value. Vendor 
supplied capacity information should be easy to verify using open source company financial, legal 
and technical records (e.g., annual reports, ITU and FCC filings). Naval peacetime theater 
requirements, the number of required accesses need to conduct routine underway operations are 
used because these requirements are fairly constant and can therefore serve as a baseline. 
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Figure 24. Notional Excess Capacity Chart 



4. Summary 

The capacity MOE is intended to reflect a system’s maximum rate of reliable information 
transmission. It is measured using three MOPs; data rate, mobility and quantity, each of which 
has sub-attributes (see Figure 25). Drop-dead, threshold and objective values are presented in 
the capacity section of Chapter IV. 



sold capacity. Furthermore, DoD would have priority use of the entire block of capacity the instant 
it is required. 
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MOE=CAPACrrY 

The maximum rate of reliable information 
transmission. 



MOP = Data Rate 



t 



MOP = Mobility MOP - Quantity 



— Transmit Data Rate 

— Receive Data Rate 



— Velocity 

— Pitch 

— Roll 

— Yaw 



— Total Accesses 

— Excess Capacity 



Notes: 

1) All MOPs assume a Standard Terminal: with a specified weight, volume, BER, location, 
environment and elevation angle. 

2) Mobility assumes a user determined fixed capacity. 

3) For Quantity a theater of operation is specified. 



Figure 25. Capacity Summary 



G. PROTECTION 
1. Defined 

Figure 26 provides the CRD and therefore the joint (JROC) agreed upon definition of 
protection as it applies to MILSATCOM. This definition not only applies to the communications 
path (i.e. the pipes) but also to the information flowing through those pipes. 



PROTECTION 



A system’s ability to avoid, prevent, negate, or mitigate 
the degradation, disruption, denial, unauthorized 
access, or exploitation of communications services by 
adversaries or the environment. 



Figure 26. Protection Defined [CRD, 1998, pp. ED*5] 
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2 . 



Why is it Required? 



It is no secret that U.S. military forces rely on information exchange and transfer as a 
fundamental warfighting capability. All of the current and future DoD doctrine publications state 
that success in warfare will be supported by information dominance. Therefore, most readers 
would inherently understand that protection of capability and security of information is a 
fundamental military requirement. As presented in Table 6, naval force protection requirements 
are analogous to the DoD requirements as presented in the CRD. The expeditionary nature and 
the ability to project power from the sea require naval forces to make extensive use of C4I 
systems. Therefore it is natural that naval forces require protected communication services in 
order to protect lives and effectively carry out their missions. 



CATEGORY 



Protection • 









RATIONAL (WHY REQUIRED) 
DoD/JOINT REQUIREMENTS 

C4I is a prime target and a center 
of gravity, which the U.S. expects 
adversaries to attack. 

Must deny adversaries the ability 
to decapitate U.S. C2 and ISR 
capabilities. 

Nuclear deterrence remains a top 
DoD priority and requires 
survivable C2. 

Must provide anti-jam, protection 
from SIGINT, information security, 
and other defensive information 
operations measures. 






RATIONAL (WHY REQUIRED) 
NAVAL REQUIREMENTS 

Expeditionary nature implies a 
reliance on C4I and therefore a 
need to protect this center of 
gravity. 

Structured to project power and 
therefore need to have protected 
communications capability between 
all warfighting elements. 
Self-supporting implies the ability to 
unilaterally act to counter a C4I 
attacker. 

Unrestricted implies the ability to 
combat or limit intrusion and attack 
of C4I intended to restrict 
operational capability. 

Scalable implies that levels of 
protection can be increased or 
decreased to meet the existing 
threat. 



Table 6. Protection Justification 
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3. 



Measuring Protection 



Given that protection is a system’s ability to "avoid, prevent, negate, and/or mitigate the 
degradation, disruption, denial, unauthorized access, or exploitation of services or resources" the 
CRD divides communication services into four categories. [CRD, 1998, pp. 4-15] These 
communication service categories are: survivable; anti-jam; low probability of intercept (LPI) and 
low probability of detection (LPD); and U.S. control. [CRD, 1998, pp. 4-15] Each of these 
measures focus on the communications path, (i.e. the pipe) and not the information traveling 
through the system. Within in the context of naval use of commercial wideband service, it is the 
author’s opinion that while important these "comms pipe" protection categories are not the 
measures that should be used to characterize the protection MOE within this hierarchy. 
Justification of this position and presentation of more appropriate MOPs is the focus of the 
remaining portion of this section. 

Survivable SATCOM is intended for use by the national strategic nuclear forces, the 
national command authority and the upper echelons of the military hierarchy. While some naval 
assets are part of the nation’s strategic nuclear forces use of commercial wideband SATCOM by 
these platforms for critical nuclear force communications links is highly unlikely. Survivable 
SATCOM is intended to provide communications capacity during and after a nuclear strike. Given 
that there is little commercial market for this type of communications and further that the cost of 
the technology involved is quite high, it is highly unlikely that commercial providers will construct 
survivable systems. This does not mitigate the naval users need for survivable communications it 
is only intended to state that use of a survivable MOP within this hierarchy provides no added 
value. That is, it is highly likely that all evaluated commercial wideband systems would receive a 
utility score of 0.0 (i.e., not survivable) under this MOP. 

Many commercial wideband vendors envision their systems will be extensions of the 
Internet extending the worldwide web to millions of international customers. In terms of system 
design drivers, commercial wideband producers are not concerned with operation of their systems 
in a traditional hostile environment (i.e., hostile actions other than information operations). Their 
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product is intended to be operated by the corporate and home user operating in a peaceful (i.e., 
not directly engaged in armed conflict) environment. Characterization of this operating 
environment is not intended to imply that hostility does not exist, only that overt physical conflict 
(i.e., jamming and destruction of hardware) is considered to have a low probability of occurrence 
and therefore is not a design driver. Instead commercial providers recognize that many of their 
customers require specified levels of confidentiality, authenticity, and integrity"*® in order to protect 
against hostile information operations. Protection against these types of threats is where naval 
wideband SATCOM evaluators should focus their attention. Furthermore, for these reasons use of 
an anti-jam MOP is not included in this hierarchy. Justification of this position follows. 

Protection against active jamming is a desired military capability, however it does not rank 
high on the commercial users list of requirements. In terms of engineering trades during system 
development it is probable that commercial designers would trade the cost of jamming protection 
for increased capacity or some other more commercially desired capability. This does not imply 
that commercial systems will not have some limited capacity to mitigate frequency or co-channel 
interference. Instead it indicates that active nulling or jamming detection to the levels desired by 
naval users will likely not be provided. For this reason evaluation of a commercial system’s 
capability to detect and counter the effects of a sanctuary or tactical jammer provides no 
discriminating input to the evaluation hierarchy. If all systems score a utility of 0.0 for jamming 
then no discrimination between systems has been provided. Again this does not imply that naval 
users should suppress their need for anti-jam capability, instead naval users should not expect 
commercial systems to meet this need. 

The protection requirement of "location covertness" as represented by the LPI/LPD 
measure refers to the ability to locate a user by intercepting or detecting the SATCOM terminal 

" Confidentiality . Bad guys can’t read our traffic. Confidentiality includes secrecy of the data 
itself and at least in the definition, traffic analysis immunity (bad guys reading the addresses and 
deriving intelligence from knowing who’s talking to whom). Authenticity . Bad guys can’t fake my 
traffic to you. Nobody forged my messages. Integrity . Can’t tamper with my traffic. What you got is 
what I really sent you.” [Buddenberg, 1995] 
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radio frequency transmissions. This need is predominantly a military exclusive need that is not 
widely (if at all) represented in the commercial community. Use of covert forces, covert sensors, 
the desire to execute surprise operations and mask force strength all contribute to the military's 
need to employ LPI/LPD systems. Similar survivability and anti-jam, measurement of a 
commercial system’s LPI/LPD ability is not likely to add value the evaluation hierarchy. First, the 
civilian user who is using a terminal to perform legal information transactions as little need for 
LPI/LPD. Second due to the high frequency and therefore narrow beam width (i.e., Ka band and 
higher) most of the commercial systems have some intrinsic LPI/LPD capability. Differences in 
detection capability could potentially stem from the power levels required to close the link between 
the earth terminal and the satellite.''^ Regardless these differences are most likely minor and 
LPI/LPD capabilities would be nearly identical. For this reason within this hierarchy an LPI/LPD 
MOP will not be used as a measure of protection. 

The CRD states the U.S. Control "usually means the system is under the direct 
operational control of a corporate, private, or government activity that is subordinate or 
immediately responsive to the national legal direction of U.S. authority." [CRD, 1998, pp. 4-19] 
Within this hierarchy levels of control and the United States’ ability to control access to a system is 
addressed under the Access and Control MCE, and the relationship and legal MOPs. For this 
reason within this evaluation hierarchy U.S. Control is not a MOP under the protection MOE. 

As previously mentioned the protection concerns that commercial vendor’s will address 
are those that the civilian market sees as important. Civilian users were identified as requiring 
various levels of confidentiality, authenticity and integrity, or stated another way information 
protection. It was suggested that naval evaluation of the "comms pipe" protection measures 
provides little value, however the analysis of a system’s information protection capability has 
significance. Historically these measures of information security have been viewed as network 

For a fixed capacity, fixed antenna size and fixed BER more power would be required to close 
the link between an earth terminal and a GEO satellite than that required for the link between a 
LEO satellite and the same earth terminal. 
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only measures that do not directly apply to a SAT COM system (i.e., the telephone company is 
only responsible for protecting the line not protecting the information on the line). That is SATCOM 
is seen as only a pipe and therefore the SATCOM provider must ensure that the pipe is protected. 
This line of thinking is short sighted and fails to recognize that SATCOM is an extension of the 
information network and security of the information flowing through the pipe is just as critical as 
security of the pipe. A possible reason for the separation of path protection from information 
protection is that communication service providers did not want to be legally liable for protecting 
user information. This "use our system at your own risk attitude" is understandable given today’s 
litigious environment, however it does not reflect the growing commercial demand for information 
security. 

It is the author’s opinion that effective information security (over SATCOM) can and will 
only be accomplished when SATCOM designers start thinking of their systems as information 
networks and not only as the "pipes that information flows through." Stated another SATCOM 
providers have historically consider themselves first and foremost as designers/providers of layer 
1, the physical layer of the ISO 7-Layer reference model. As a result SATCOM providers give less 
attention to the six remaining layers and fail to fully exploit the information security capabilities that 
other layers can provide (refer to Figure 27). 

Layer 7 Application 
Layer 6 Presentation 
Layer 5 Session 
Layer 4 Transport 
Layer 3 Network 
Layer 2 Data Line 
Layer 1 Physical 



Figure 27. The Historic ISO 7-Layer Reference Model [Comer, 1997, pp. 157] 
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By adapting the attitude that SATCOM service is more than a “comms pipe," designers 
are then free to make trades across layers of the ISO model and subsequently design a system 
that optimizes information security. For example, effective security measures that are employed at 
the transport layer (layer 4) maybe driven by physical layer design. It is understood that the ISO 
model was developed to enhance integration by creating definable boundaries and interfaces. 
However excessive compartmentalization while increasing "within layer" innovation has reduced 
cross-layer optimization, and as a result overall solution optimization has suffered. As a general 
rule of thumb "the higher the layer (ISO model) at which you can gain appropriate security service, 
the less you have to depend on the network to provide the service." [Buddenberg, 1995] The 
following quote further demonstrates this concept: 

For example, with secure email -- an Application Layer implementation of a 
security service - all security functionality is provided in end systems, none is 
required of the network infrastructure (links, routers, gateways, etc). This means 
that it is not necessary to own or control the network in order to have secure 
service. In concise terms, with application layer security we have confidentiality 
and authenticity over untrusted networks. [Buddenberg, 1995] 

What the quote implies is that by creating an open network that allows for application 
layer security, information can be successfully protected. It is within this context that the proposed 
hierarchy evaluates protection using three MOPs; confidentiality, authenticity, and integrity. The 
goal of the MOPs is to highlight design concepts that incorporate multi-layer information 
protection. 



a) Confidentiality 

Confidentiality extends from protecting transmitted information to protecting users 
from traffic analysis and in this case protecting the user’s geo-location. Although encryption can 
be used to protect the confidentiality of transmitted information it does not protect the user from 
traffic analysis. Significant intelligence can be collected by a user who can monitor who talks to 
whom, and at what time the information exchange occurs. It is desired that commercial systems 
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allow naval users to employ encryption (at the application layer) as well as allow users the ability 
to protect and hide their addresses. Address hiding is a potential sticky point for commercial 
providers in that it maybe a primary method for user billing. Despite this problem naval users need 
to remain autonomous and should not have their traffic patterns analyzed. A final aspect of 
confidentiality is the desire to protect the user’s physical location. As a communication network 
that employs moving transmitters and receivers (both user terminals and spacecraft) SATCOM 
systems require accurate geo-location to efficiently close the communications link. Accurate 
customer billing Is also another reason for geo-locating every user terminal. A standard method 
for tracking user location is to create a geo-location database or at least a file that provides the 
location of various user terminals. Exploitation of this geo-location data can be detrimental to 
naval forces regardless of whether forces are trying to remain covert or not. A significant amount 
of intelligence can be obtained from an accurate list of the location of each and every naval 
terminal within a given theater. Therefore confidentially encompasses data secrecy, traffic secrecy 
and geo-location secrecy. 

Measuring how a vendor intends to protect a user’s confidentiality is difficult and 
therefore it is proposed that the confidentiality MOP be reflected as a subjectively determined 
value. Users would establish a confidentiality specUum somewhat like the one presented in Figure 
28 . 



A low degree 
of 

Confidentiality 



Encryption is not 1 

authorized. 

Address information “ 

and user profiles are i 

sold to external J 

parties. 

Geo-location data is | 

unprotected and j 

exploitable- 




Encryption authorized. 
Traffic analysis and 
address information 
only accessible to 
employees of the 
vendor. 

Access to the geo- 
location data base is 
moderately controlled. 





• Entirely closed 
system with limited 
and controlled 
access to user 
address and geo- 
location database. 

• Users are notified 
when inquires are 
made about them. 



Figure 28. Confidentiality Spectrum 
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The intent of this spectrum is to allow a group of informed users to categorize a 
system’s ability it provide confidential service. This measure is intended to be subjective and 
therefore allow evaluators to discriminate between competing systems. The boundaries presented 
in Figure 28 are not all inclusive and are certainly open to debate. Recall that the goal of 
confidentiality is to characterize a systems ability to protect information travelling through the pipes 
not the pipes themselves. 

b) Authenticity 

Authenticity reflects a system’s ability to ensure that hostile users can not forge a 
naval user’s message (spoofing). The goal is to protect user identify and increase the faith that 
users at both ends of an exchange are actually talking to who they think they are talking to. In 
general authenticity can be accomplished at the network layer, transport layer and the application 
layer. [Buddenberg, 1995] Again measurement of an "authenticity capability" is difficult to quantify 
and therefore evaluators should subjectively attempt to identify how the vendor intends on tackling 
the authenticity problem. Subjective evaluation should be derived from a vendor presentation in 
which the vendor articulates how authentication will be addressed within their system. The vendor 
would then receive an authenticity score based on a user derived authenticity spectrum similar to 
the confidentiality spectrum. Elements that should be included in an authenticity spectrum are: 

• The protection of billing databases that associate a user’s SATCOM address with his 
actual identification. 

• Use of application layer authentication keys. 

• Implementation of procedures that prohibit dual use of user IDs. 

• Implementation of procedures that notify authorized users that their ID may have 
been compromised. 

c) Integrity 

Integrity attempts to capture a system’s ability to ensure that a user’s message is 
not tampered with enroute to the intended destination and that what the sender transmitted is 
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indeed what the receiver received. [Buddenberg, 1995] Protection against message modification 
can be accomplished at various levels of the network and is closely coupled with confidentiality 
and authenticity. The ability to modify a user’s transmitted message implies that the user’s 
confidentiality and authenticity has been compromised. Measurement of message integrity or 
rather a system’s ability to ensure message integrity should also be a subjectively determined 
user value. As with confidentiality and authenticity evaluators should debate and subsequently 
create an integrity spectrum. The spectrum should equate a vendor’s ability or inability to provide 
adequate integrity security to a numerical score. 

4. Summary 

Commercial systems are not likely to provide survivable, anti-jam, or LPI/LPD 
communications and therefore within this hierarchy these factors are not considered as protection 
MOPs. Instead the focus is on information protection, and the intent is to measure a system’s 
ability to protect the information flowing through the SATCOM pipes. It was within this line of 
reasoning that three information protection performance measures were presented; 
confidentiality, authenticity and integrity (Figure 29). In an effort to not restrict future evaluations 
specific quantitative values for these measures were not determined. Instead each measure was 
presented as a spectrum of capability in which capability is subjectively determined by the 
user/evaluator. 

Protection 

I I 1 

Confidentiality Authenticity integrity 

Figure 29. Protection MOPs 

H. INTEROPERABILITY 

1 . Defined 

National Military Strategy as well as joint vision and naval vision documents recognize that 
most future military operations will be joint and/or combined operations. Shared information 
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promotes a “common interpretation and understanding of the battlespace fundamental to ensuring 
unity of effort and synchronization of action," and therefore is critical to effective joint operations. 
[CRD, 1998, pp. 4-38] Interoperability enables information sharing. As indicated in Figure 30 
interoperability refers to a force’s ability to communicate with others in a mutually beneficial 
manner that increases overall combat effectiveness. Interoperability extends beyond the naval 
forces and the other military services and includes non-DoD agencies as well as allies and 
potential coalition partners. 



INTEROPERABILITY 



The ability of systems, units, or forces to provide 
information services to, and accept information 
services from, other systems, units, or forces and 
then to use the services so exchanged to enable 
them to operate effectively together. 



Figure 30. Interoperability Defined [CRD, 1998, ES-8] 

2. Why is it Required? 

As already mentioned future conflicts will rarely involve a single U.S. service. More than 
likely operations will be executed by a joint force operating in cooperation with allied and/or 
coalition forces. [CRD, 1998, pp. 4-38]. Furthermore, DoD has placed significant effort into 
developing interoperability policy to ensure that current and future acquisition of information 
technology systems will provide "the needed seamless interoperability." [JTA V2.0, 1998, pp. iii] 
The recently published Joint Technical Architecture Version 2.0, (JTA V2.0), is an information 
technology (IT) policy document that is produced by a team consisting of all DoD components as 
well as components of the intelligence community. [JTA, 1998, pp. iii] In its scope JTA V2.0; 

Mandates the minimum set of standards and guidelines for the acquisition of all 
DoD systems that produce, use, or exchange information. 

Shall be used by anyone involved in the management, development, or 
acquisition of new or improved systems within DoD. [JTA, 1998, pp. 1-3] 
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Furthermore the reasoning that went into defining JTA standards matches naval force 
interpretability requirements. That is the JTA states that achievement and maintenance of a 
seamless information environment requires interoperability: 

• Within a Joint Task Force/Commander in Chief (CINC) Area of Responsibility 
(AOR) 

• Across CINC AOR boundaries. 

• Between strategic and tactical systems. 

• Within and across Services and Agencies. 

• From the battlefield to the sustaining base. 

• Between US, Allied, and Coalition forces. 

• Across current and future systems. [JTA, 1998, pp. 1-3] 

JTA requirements are periodically reviewed and updated and are intended to ensure the 
procurement of "the envisioned objective of a cost-effective, seamless integration environment." 
[JTA, 1998, pp. 1-3] The bottom line is that JTA V2.0 is applicable to the procurement of 
commercial wideband SATCOM intended for naval use (see Table 7). As a complementary 
document JTA requirements are consistent with other DoD programs and initiatives including the 
Defense Information Infrastructure Common Operating Environment (Dll COE). [JTA, 1998, pp. iii] 
Therefore adherence to JTA requirements provides a good measure of a wideband system’s 
interoperability. 

Although the JTA V2.0 describes and mandates standards for procurement of DoD 
systems it does not and can not enforce or require interoperability among competing commercial 
vendors. It is possible that since JTA V2.0 attempts to set standards that are "stable, technically 
mature, and publicly available" that conformance to JTA may inadvertently encompass some of 
these external requirements. Any capacity (i.e., a third party terminal that provides multi-vendor 
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compatibility) that increases a commercial system’s ability to interoperate with other vendor 
systems has potential value to the naval user. 



CATEGORY 



Interoperability • 



RATIONAL (WHY REQUIRED) 
DoD/JOINT REQUIREMENTS 

MILSATCOM is the space portion 
of the Defense Information 
Infrastructure. 

Most operations are joint in nature 
and execution (Land, Air, Naval, 
Mobility, Combat Support and 
Special Operations Forces). 

US Forces conduct missions with 
Allies, Coalition Partners, and 
Government Agencies. 
Warfighters use a variety of 
communications to effect needed 
information transfers. 



RATIONAL (WHY REQUIRED) 
NAVAL REQUIREMENTS 

Expeditionary nature requires the 
ability to act instantaneously without 
delay caused by information 
translation or communication system 
“work arounds”. 

Structured to project power using a 
variety of different assets that rely on 
effective information exchange to 
increase combat effectiveness. 
Self-supporting requires the ability to 
share logistics and supply 
requirements with all layers of 
support hierarchy in a timely fashion. 
Unrestricted implies that information 
exchange is not constrained by 
unique hardware or software 
technologies. 

Scalable implies that information 
exchange between all participants is 
possible at all levels of conflict. 



Table 7. Interoperability Justification 
3. Measuring Interoperability 

Measurement of interoperability should be discrete. That is a commercial system should 
be categorized according to its full capabilities not its partial capabilities. For example if a system 
conforms to a majority of JTA standards but not all JTA requirements then the system is not JTA 
compliant. In terms of interoperability, degrees of conformity do not apply. A system that is 
advertised to meet 75% of DoD’s interoperability requirements is not an interoperable system. 
Given this reasoning only one measure of interoperability is needed. Therefore within this 
hierarchy the interoperability MOE does not have any MOPs and the MOE itself is the 
performance measure. 
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I. ACCESS AND CONTROL 

1. Defined 

Webster’s dictionary defines access as "the right to enter or use." In terms of SATCOM 
the right to use has two elements; the ability to physically connect to the system (i.e. receive the 
downlink), and the authorization by a controlling authority to use the system. Recall that the 
coverage and capacity MOEs addressed the connection part of access. The coverage MOE had 
four MOPs that defined when and were access could occur. While the capacity MOE relates how 
much service a system can provide once connected. The access and control MOE is used to 
account for the permission to use the system part of the definition. Naval warfighters firmly believe 
that regardless of a communication system’s technical capabilities it has no value if there is no 
assurance that permission to use the system will be granted when it is needed. For this reason 
assured access to SATCOM is a fundamental and primary need of warfighters. [CRD, 1998, pp. 
4-31] Within this hierarchy access reflects the permission aspect of the definition and is defined, 
as the authority to use SATCOM service when and where required, (refer to Figure 31). MOPs 
that reflect the level of assured access or certainty that access will be granted are presented in 
the Measuring Access and Control sub-section. 

Another fundamental naval force need is have the authority and ability to responsively 
"apportion and reapportion" SATCOM resources. [CRD, 1998, pp. 4-31] This authority and ability 
to dictate resource utilization is called control. SATCOM control has multiple levels of meaning 
and can imply the authority to grant access to system resources on up to having the ability to 
command individual satellites. The CRD defines control as “the ability and mechanisms needed to 
effectively plan, monitor, operate, manage, and manipulate the available MILSATCOM 
resources," (refer to Figure 31). [CRD, 1998, pp. 4-31] This definition implies the ability to dictate 
what occurs in all aspects of the MILSATCOM domain. Potential overlap with the protection MOE 
is clearly evident, however this hierarchy attempts to limit control to control over resource 
allocation, while protection covers information and system security measures. The MOPs 
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presented in the Measuring Access and Control sub-section should make this difference more 
visible. 



ACCESS 



The authority to use SATCOM service when and 
where required. 



CONTROL 

The ability and mechanisms needed to effectively 
plan, monitor, operate, manage, and manipulate 
the available SATCOM resources. 

Figure 31. Access and Control Defined 
2. Why is it Required? 

As previously stated a fundamental naval force need is assured access to and control of 
SATCOM resources. Table 8 translates the DoD access and control needs as presented in the 
CRD to naval force access and control needs. The single pillar that supports the access and 
control needs is represented by the standard saying, "fight as you train." Naval warfighters can 
train and patrol in peacetime making effective use of commercial SATCOM, however that same 
capability must be immediately and continuously available throughout any and all hostile 
encounters. At the minimum loss of a capability that was extensively relied on during training will 
create operational inefficiencies, and at the maximum warfighters will die. Assured access is a 
fundamental SATCOM requirement. 
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CATEGORY 



Access and 
Control 



RATIONAL (WHY REQUIRED) 
DoD/JOINT REQUIREMENTS 

• Access to information and 
communications on-demand: 
fundamental need of the 
warfighter. 

• Warfigthers must have control 
over their information and 
MILSTCOM domains. 

• Resources be rapidly and 
dynamically reconfigured to 
respond to changing operational 
situations and priorities. 



RATIONAL (WHY REQUIRED) 
NAVAL REQUIREMENTS 

• Expeditionary nature requires 
assured access and control in order 
to support forward deployed 
dispersed forces. 

• Structured to project power requires 
the authority to grant access to and 
maintain control of SATCOM assets 
during direct hostile action. 

• Self-supporting implies capability to 
dynamically configure and 
reconfigure SATCOM architecture. 

• Unrestricted by international or 
political regulations that limit the 
force’s expeditionary or power 
projection capabilities. 

• Scalable implies the ability to 
apportion and reapportion SATCOM 
assets as dictated by the operational 
situation'. 



Table 8. Access and Control Justification 
3. Measuring Access and Control 

SATCOM access and control are rudimentary naval warfighter needs and should drive the 
selection of alternative concepts. Furthermore, evaluation of commercial systems must highlight a 
system’s ability or inability to meet these two needs. The desire to "jump on the commercial band 
wagon" should be tempered by the user’s valid need for access and control. Clearly full naval 
control of a commercial system is not likely, however this does not imply that commercial systems 
have little value. Naval use of commercial SATCOM has occurred in the past, is currently 
occurring and will continue to occur in the future. What system evaluators must do is clearly 
articulate the risks (in terms of access and control) involved in the use of a particular commercial 
SATCOM provider. Once articulated, user (i.e., operational warfighter) opinion through MOE and 
MOP weighting must be honored as the ultimate measure of value. Although politically difficult, 
compromise on SATCOM access and control beyond the recommendations of experienced 
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operators can threaten the lives of naval warriors. Operators clearly understand that trades and 
compromises in capability are fundamental to achieving success and therefore further recognize 
that despite potential short comings in access and control commercial SATCOM can play a vital 
role in battlefield success. 

The goal to be accomplished in measuring access and control is not to restrict the 
solutions that can be evaluated, instead the desire is to clearly articulate the relative risk that 
different alternatives possess. Warfighters understand risk, accept that risk exists, and train to 
operate in a risk filled environment. Once relative risk is identified and understood naval operators 
will adapt to it and identify ways to work within the acknowledged level of risk. Identifying what 
level of access and control risk a system possesses is the output of the access and control MOE. 
Two MOPs are used to represent access and control risk, each of which is presented and defined 
later in this section. 

It should be clearly evident that in order to be profitable commercial SATCOM providers 
must possess the same level of access and control that DoD desires. The vendor must be able to 
authorize access and maintain the ability and mechanisms needed to effectively plan, monitor, 
operate, manage, and manipulate the available SATCOM resources. In as much as access and 
control are fundamental naval force needs they are also service provider needs. The pillar 
supporting commercial access and control needs is profit, generated by providing quality service. 
Quality service critically depends on a corporation’s access and control capabilities. For these 
reasons it is highly unlikely that commercial vendors will give up access and control to U.S. forces. 

Both access and control can be evaluated at different levels (regions). The region that 
should be evaluated within this hierarchy is the interface between customers and the commercial 
vendor (see Figure 32). The customers own the level below this interface, and the region above 
resides with the commercial vendor. As the previous paragraph describes the SATCOM provider 
will retain access authority and resource control for the satellite system as a whole. The provider 
must maintain this control in order to operate a quality system that continues to retain and attract 
customers. Within the Vendor Region the provider positions individual satellites, tracks customer 
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billing, activates and deactivates customer accounts, reconfigures system components in reaction 
to failures and performs any of the other required control activities. The naval user should not 
expect direct access to the Vendor Region. 



Vendor Region 

Vendor maintains access authority and resource controi. 



interface Region 

Area where evaluation hierarchy shouid focus. 



Customer Region 

Naval Ck>mmander maintains Others maintain access authority 
access authority and resource and control their own resources, 
control. 



Figure 32. Access and Control Regions 

On the other hand naval forces will maintain access authority and resource control over 
the users and components that reside in the naval customer area of the Customer Region. All of 
the (naval) leased or purchased terminals and the users associated with those terminals reside in 
this region. Within this hierarchy it is assumed that naval forces have an overarching resource 
management and control mechanism that provides the ability to authorize access to and 
apportion, reapportion, configure, and reconfigure assets. This implies capabilities analogous to 
the current management of commercial telephone service throughout DoD. That is naval resource 
managers determine user priorities, allocate capacity, deny service to unauthorized (naval) users, 
and perform a whole host of other network management responsibilities. Naval force authority 
does not extend into the "other customer" section of the Customer Region. Other customers, quite 
possibly threat countries, maintain access authority and resource control within their specified 
regions. 

There remains one region to discuss, the Interface Region. No specific equipment or 
hardware capability exists within this region. Instead this area accounts for the political and legal 
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factors that influence access and control. As the term interface implies, these political and legal 
parameters affect both the Customer and Vendor Regions. It is the author’s opinion that the only 
way to influence control or access authority over a commercial system is to do so through the 
Interface Region. That is political and legal action is the mechanism by which naval forces will 
gain (loose) access authority and resource control capability. Characterization of these political 
and legal factors is the focus of the access and control MOPs. 

Due to the high cost of wideband SATCOM constellations most proposed and emerging 
ventures are internationally funded. International SATCOM consortiums enhance and provide 
access to foreign markets, assist in the resolution of foreign legal restrictions, and share 
development risk over a wider base. Due to their international makeup, politics and legal 
regulations drive consortiums and therefore the risk associated with doing business with a 
consortium is also driven by political and legal factors. For this reason two MOPs, relationship to 
the U.S. and legal restrictions are used to characterize the access and control MOE. 

The intent of the relationship MOP is to characterize the risk associated with a particular 
vendor due to the type of relationship that the owners (i.e., consortium members, major stock 
holders, associated financial institutions) of the service have with the United States military. 
’"Owners" is intentionally a very broad term and indicates a desire to include all people, 
organizations or countries that have significant influence over the SATCOM vendor. Classification 
of a provider’s relationship to the U.S. military is a multi-variable problem that does not have a 
clear-cut template style solution. For use within the hierarchy relationship should be thought of in 
terms of a relationship spectrum, where potentially hostile corporations are on the left and entirely 
U.S. friendly corporations are on the right. For some it maybe valuable to divide the spectrum into 
three primary regions; high risk, medium risk and low risk. High risk implies a consortium that 
resides on the left third of the spectrum and implies that the probability of access denial and loss 
of control are very high. Figure 33 provides a graphic representation of the relationship spectrum. 
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Probability of access denial and loss of control 



High Risk 




• No U.S. owner is 
involved or U.S. 
owners do not have a 
controlling interest. 

• Owners are 
traditional enemies of 
the U.S. 

• Consistently oppose 
U.S. international 
policy. 

• Encourage terrorist 
or overt military acts 
against the U.S. 
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economic 1 
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U.S. i 

• Occasional | 

supporters of U.S. 

policy and rarely 
overtly oppose U.S. 
interests. 

• At least one owner is 
an U.S. company. 

• U.S. owner is a 
controlling partner. 



^ 

Low Risk 

• Exclusively U.S. 
owned and operated. 

• Owners are 
traditional allies and 
strong supporters 

• Owners have military 
and economic 
alliances with the 
U.S. 

• Consistent 
supporters of U.S. 
international policy 



Figure 33. Relationship Spectrum 

Although the author has provided adjectives to assist in relationship classification it is 
critical to realize that this is not intended to be a clear-cut template. The boundaries between 
areas are gray. Numerous resources can be used to assist in classification of a corporation. 
Products produced by the Defense Intelligence Agency, the State Department, the Central 
Intelligence Agency and any of the other government agencies whose job it is to understand and 
characterize other nations and international organizations can be helpful in defining the 
relationship spectrum. 

The relationship MOP attempts to reflect the risk involved with the political part of the 
Interface Region. For example, a vendor that was classified as a staunch friend of the U.S. may 
willingly, (through intense political pressure and possibly monetary incentive) disallow service to 
existing customers (i.e., threat countries) during a time of U.S. military crisis. It is important to 
recall that a corporation in a particular country is in most cases an independent organization (i.e., 
not state run) and national policy rarely dictates the corporation’s actions. However when a vendor 
resides in a country that is a staunch U.S. ally there is an increased probability that the country 



also has troops committed to the same military crisis. In this case the corporation will be under 
significant political pressure to cooperate with U.S and allied forces. At what price that cooperation 
is achieved is another unknown element of risk. It is critical to point out that the political process is 
a two way street and pressure can be applied to deny U.S. access and control. There in lies the 
value of the relationship MOP, in that it should be used as a predictor of negative political 
influence. 

The legal MOP attempts to characterize the risk associated with contractual regulations of 
a particular service provider. Similar to the relationship MOP the legal MOP is presented as a risk 
spectrum (see Figure 34). Legal measures that restrict the effective operational use of the system 
during all phases of military action (peace through hostile action) are considered high risk and 
probability of access denial and loss of control are high. These high risk regulations are 
represented on the left-hand side of the spectrum. Naval friendly legal measures are represented 
on the right-hand side of the chart. As with the relationship MOP the subdivisions presented in 
Figure 34 are not intended as a template but as a point of departure for evaluator discussion. 
There are numerous combinations of legal measures that can be evaluated however three 
overarching legal concerns must be considered. These three concerns are: 1) restrictions on war 
time use (i.e., a war clause); 2) restrictions on frequency use for mobile users; and 3) restrictions 
on landing rights. 
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Significant Mission 
Hindering Restrictions 




Tolerable Legal 
Constraints . 



1.0 <* 



Probability of access denial and loss of control 



^ 0.0 



High Risk 




• War clause stating 
that use of service 
during overt and 
covert action is 
prohibited. 

• Mobile use is not 
authorized. 

• The customer must 
negotiate host nation 
approval on his own. 






Medium Risk 






• Host nation approval 
has been pre- 
negotiated, but is not 
guaranteed. 

• Mobile user is 
"secondary" user of 
assigned spectrum. 

• No immediate 
termination of service 
during hostile action 
however continuation 
of service is 
contingent on 
consortium decision. 






I 



Low Risk 

• No war clause, or 
restriction on use 
during overt and 
covert action. 

• Mobile user is 
"primary" user of 
assigned frequency. 

• Host nation approval 
has been negotiated 
by provider and 
guaranteed up front. 



Figure 34. Legal Spectrum 

A war clause includes any legal terminology that restricts access or message content 
during time of overt or covert military action. It is obvious that naval forces desire not to have to 
switch communications systems as they transition to hostility. Any restrictions on message 
content (i.e., calls or orders to engage a hostile target) or service access dramatically reduces a 
system’s contribution to combat effectiveness. However, if the system is relied on for providing 
non-mission critical service (i.e., tv to sailors) then these restrictions maybe acceptable. The 
bottom line is that a war clause is unacceptable for a system that is going to provide mission 
critical communications. Furthermore, the specific and detailed definition of what type of message 
traffic is allowed and disallowed must be clearly evident and uniformly interpreted. 

The second legal restriction is the constraints imposed by international and national 
regulatory agencies. The primary international regulatory agency is the International 
Telecommunications Union (ITU) and within the U.S. the national regulatory agency is the Federal 
Communications Commission (FCC). Both of these agencies place constraints on satellite 
communications providers in an effort to assure reasonable protection from interference. [Vuong, 
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1997] It is important to realize that most nations have a national electromagnetic spectrum 
regulatory agency that performs duties analogous to the FCC. It is often the case that national 
regulations are not only stricter than ITU regulations but what is restricted in one country is not in 
another. This fact dramatically increases a service provider’s legal concerns. However, in general 
adherence to ITU standards is a good indicator of international acceptability. 

Of primary importance is how the service provider has registered the system with the ITU. 
As a regulatory body of the United Nations one of the ITU’s tasks is to authorize electromagnetic 
spectrum use for different types of satellite service in order to prevent communication system 
interference. The ITU has defined different types of satellite service and associated frequency 
bands and priorities to each service type. There are three service types that are of particular 
interest to the naval force user; fixed satellite service (FSS), broadcast satellite service (BSS) and 
mobile satellite service (MSS). Priority of service implies that within a given frequency band one 
service category is considered primary and others are classified as secondary and authorized to 
operate on a not to interfere basis. It is important that naval evaluators clearly understand the 
difference between the three primary service categories mentioned above. 

BSS is a radiocommunication service in which signals transmitted or 
retransmitted by satellites are intended for direct reception by the general public. 

FSS is a radiocommunication service between earth stations at specific fixed 
points when one or more satellites are used. 

MSS is a radiocommunication service between mobile earth stations and one or 
more satellites, or between satellites used by this service or between mobile earth 
stations by means of one or more satellites. [Vuong, 1997] 

In general terms BSS is used for service such as DirectTV in which the user has a receive 
only terminal and constraints are therefore placed on the transmitting satellite. FSS is primarily 
used for communications between fixed earth terminals and GEO satellites. For this service 
restrictions are placed on both the transmitting satellite and the earth terminal transmitter. Given 
that GEO satellites either appear motionless for move very little relative to a position on the earth 
terminal transmitter restrictions focus on limiting antenna gain patterns. MSS implies that the earth 
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terminal is capable of motion as well as two-way communications. Because the earth terminal can 
move frequency interference is more difficult to predict. Historically, due to antenna technology 
limitations FSS was the dominant form of service and significant international infrastructure has 
been constructed to support FSS. The recent growth of the mobile user market coupled with 
increased demand for an ever more congested spectrum has put MSS in direct conflict with FSS. 

As indicated the service definitions were created to assure a reasonable amount of 
interference protection, they do not assure interference free communications. However, "because 
no additional radio frequency interference is generated, it should be acceptable (by the ITU) to 
use: 

• FSS and MSS frequencies for BSS, 

• MSS frequencies for FSS or BSS, 

• BSS frequencies for receive-only FSS or receive-only MSS, and 

• FSS downlink frequencies for receive only MSS." [Vuong, 1997] 

What is important to gain from the listing of acceptable service is that it is unlikely the ITU will 
allow other service combinations. [Vuong, 1997] Exceptions to these rules have been granted, 
however the most critical component of the discussion is the realization that two way, (i.e., full 
duplex, transmit and receive capability) MSS is not an acceptable use of FSS frequencies.^® 
Therefore, MSS conducted on FSS frequencies is a secondary service and can only be conducted 
on a not to interfere basis. This point is critical because most of the emerging wideband 
commercial systems are filed as FSS providers operating in the Ka band (a FSS designated 
band). As mentioned exceptions to these rules have occurred (without ITU approval). For 
example, the Navy’s use of Intelsat’s C-band global transponders for the Challenge Athena 

Unless international regulations are modified the following is true under current ITU regulations. 
"Inorder to provide communications from mobile earth stations (i.e., naval planes or vessels), it is 
necessary to use MSS frequencies to comply with ITU’s rules and regulations." Furthermore, "the 
MSS frequencies available commercially (now or in a foreseeable future) are very limited in 
bandwidth (only available at VHF/UHF, L-band, and S-band) and are all used for “off-the-self" 
services that provide only voice and low bit rate (<64 kbps) data communications." [Vuong, 1997] 
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Project, which is by definition a MSS, was authorized by Intelsat and COMSAT (not the ITU) on a 
non-interference and experimental basis. [Vuong, 1997] Obviously there is inherent risk 
associated with developing warfighter reliance on non-approved “illegal" communications 
systems. In the case of Challenge Athena interference with other users while operating in the 
open ocean is not a significant concern, however the potential for interference increases when 
littoral operations are required. Given the naval force focus on littoral warfare this legal risk can be 
quite significant and must be accounted for when evaluating wideband systems intended for use 
by naval forces. It is the author’s opinion that naval forces are inherently mobile, their mission 
demands it. Attempts to mitigate this mobile characteristic to improve a FSS provider’s evaluation 
score should not occur. Use of FSS in a MSS mode must be taken for what it is; a calculated risk. 

The final legal hurdle that international SATCOM users must contend with is that of host 
nation approval. As previously mentioned most nations have some form of national 
electromagnetic spectrum and communications systems regulatory agency. The motivations of 
each agency vary from country to country and include such topics as; spectrum control for 
collection of tax revenue, control of communications systems to ensure political stability (i.e., 
authoritarian governments), and restrictions requiring domestic participation in all foreign 
ventures. Regardless of the motivation host nation approval (HNA) is required prior to operation 
and generally includes three things; 

• landing rights; refers to the placement of terminals on host nation soil 

• site operations licenses: approval documents for installing and operating terminals 

• public switch telephone network (PSTN) connection approvals: which may provide critical 
links to the. other communication systems. (Vuong, 1997] 

Accomplishment of HNA often requires presentation of numerous supporting documents 
detailing possible frequency interference problems and system capabilities. It is a significantly long 
process that can take years to complete. In this instance established commercial providers or 
providers with an internationally diverse consortium can speed the process up. Furthermore, the 
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ideal commercial provider would minimize individual customer involvement in the HNA process 
and handle the entire action by itself. Given that naval force doctrine calls for global mobility and 
the capability to project power anywhere in the world at anytime acquiring HNA must be done far 
before a projected need occurs. Evaluation of what a vendor supplies in terms of HNA provides 
insight into the potential legal risk of using that provider. 

4. Summary 

In summary, military use of commercial SATCOM implies acceptance of access and 
control risk. When using a commercial provider it is unlikely that DoD will have total access or the 
ability to control all SATCOM resources. Therefore, DoD is forced to inadvertently and indirectly 
control access through political and legal means. The predicted ability or inability to politically or 
legally influence a SATCOM provider is the suggested way for measuring access and control risk. 
Lastly it is important to realize that there is no templated definition of access and control risk, the 
users and evaluators must classify systems based on a subjective categorization of what a vendor 
can provide. A graphical representation of access and control is provided in Figure 35. 




Figure 35. Access and Control Summary 



J. QUALITY OF SERVICE 
1 . Defined 

As indicated in Figure 36, quality of service (QoS) reflects a system’s ability to conduct 
responsive and accurate information exchange. A fundamental concept is that QoS is service 
quality as determined by the user, not the vendor. The user must choose metrics that accurately 
reflect quality desires. However selection of metrics is difficult because within a wideband system 
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numerous design parameters affect QoS. The focus in this hierarchy is to identify a few of the 
primary metrics and encourage future evaluators to add or remove measures that better reflect 
the given situation. Furthermore, there is significant potential for overlap between QoS and 
interoperability. Within the gray area between the two MQEs evaluator’s should identify QoS 
measures as those measures that do not specifically define interface standards. QoS focuses on 
the service capabilities and constraints unique to a particular system. In essence the goal of the 
QoS MQE is to quantify vendor service, just as customers rate the quality of commercial cable 
television or Internet service providers. 



Quality of Service 

The ability of satellite systems to adequately 
conduct required information transfer in a timely 
and accurate manner. 

Figure 36. Quality of Service Defined [CRD, 1998, pp. 4-47] 

2. Why is it required? 

Given their expeditionary nature naval forces must have faith that the C4I systems that 
they use will provide a consistent level of service when and where ever it is required (see Table 
9). Qf course anomalies in service will occur, however repeated outages, corrupted messages, or 
consistent congestion reduces customer faith. Loss qf faith due to poor quality forces warfighters 
to seek alternatives that may reduce expeditionary capabilities. Furthermore, in order to project 
combat power users expect wideband systems to provide accurate, unambiguous, and 
responsive mission critical information. For example, congestion that increases the probability of 
loosing a message or increases the probability that a corrupted message will be delivered reduces 
customer faith. Reduced faith constrains the force’s ability to project power. At the extreme 
corrupted or delayed information could cost the lives of combat forces relying on the system to 
provide accurate and responsive command and control connectivity. Given the desire to enhance 
expeditionary capability forces also need the ability to predict reduction in quality due to weather. 
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This is especially important in littoral regions where weather is affected by both the sea and 
coastal landmass. It is recognized that due to their short wavelengths the Ka and higher frequency 
wideband systems are impacted by precipitation however this does not mitigate the user’s need to 
know and understand rain effects. Therefore in order to establish the boundaries for unrestricted 
operations the user must know and understand system weather constraints (i.e., loss of quality 
due to rain fade). In the case of commercial SATCOM where the provider owns and operates the 
satellite the military user does not need to be immediately concerned about space weather. 



CATEGORY 



Quality of 
Service 



RATIONAL (WHY REQUIRED) 
DoD/JOINT REQUIREMENTS 

• Supported warfighting and 
combat support systems drive 
performance criteria. 

• Information must be transferred 
accurately and unambiguously. 



RATIONAL (WHY REQUIRED) 
NAVAL REQUIREMENTS 

Expeditionary nature implies the 
expectation of a consistent level of 
service when and where required. 
Structured to project power requires 
accurate, unambiguous, and 
responsive mission critical information. 
Self-supporting implies a high quality 
system in which there is little need to 
interact with vendor on quality issues. 
Unrestricted by weather and natural 
phenomena within accepted 
standards. 

Scalable implies graceful degradation 
and restoral. 



Table 9. Quality of Service Justification 

The nature of naval forces and the way that the NSS intends to employ these forces often 
results in naval forces being the first on the scene. Once on scene other forces come into the 
area of operation and information needs expand. A scalable system that increases and decreases 
in capability in a graceful manner is invaluable to maintaining effective military force integration. 
Furthermore, if an anomaly should occur and service is discontinued the system should be able to 
be restored in a responsive and appropriate manner. Finally, the last naval force characteristic to 
be addressed, self-supporting, implies that naval forces should not be encumbered by poor quality 
systems that require placement of numerous trouble calls to the vendor. 
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3. Measuring Quality of Service 

Ideally QoS would be evaluated with operational testing however this may not be possible 
given the immaturity of the commercial wideband technology at this time. With this constrain and 
since the focus of this thesis is to evaluate emerging concepts as they are represented on paper, 
performance measures that anticipate quality service must be developed. The author proposes 
that the following six performance measures be used: 

• Availability, reflects the probability of getting a "dial tone" (i.e., not busy). 

• Delay, reflects the probability that no delay in information exchange occurs. 

• Blockage: reflects the probability that no blockage of information occurs. 

• Robustness: a binary measure that reflects the existence of an automated customer 
priority scheme. 

• Data Integrity, the advertised BER. 

• Weather, reflects the intensity of rain that causes mission critical outages. 



Although the measures presented above are not exactly identical to those presented in 
the QoS portion of the CRD, the reasoning behind their selection is the same. As stated in the 
CRD-. 



Timeliness of information and immediacy of access are critical factors in ensuring 
combat effectiveness in the modern, dynamic battlespace. Much of battlefield 
information’s’ value is highly perishable and critical. Target acquisition and 
engagement opportunities are becoming especially fleeting and of limited duration 
and must be acted on immediately upon being offered. Warning information must 
be transmitted immediately to be of any use. [CRD, 1998, pp. 4-48] 



a) Availability 

Simply put when a user desires access to the system, the system should be 
available. The capability being measured here is identical to a customer’s desire to hear a dial 
tone every time he picks up the phone or a "web-tone" every time he attempts to access the 
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Internet. Users must have faith that the system is available. It is assumed that the user is a 
qualified user (i.e., the access authority has authorized the user) and is within geographic and 
time coverage of the system. Within this hierarchy availability is measured as the probability of 
access, P(access). This measure should be provided by the vendor and reflects his contractual 
responsibility to provide an assured probability of access. The goal here is to have a vendor 
guarantee a probability of access regardless of system congestion. The military user needs to 
have faith (i.e., 98% probability of access) that the system will be available during periods of high 
network traffic. The military user is not concerned how the vendor will actually ensure that 
availability is achieved (i.e., bring more systems on line to handle surge demand) only that access 
is assured at a certain probability. Achievement of assured access during high traffic times may 
require enforcement of a priority user scheme (refer to the sub-section on Robustness), where 
lower priority users are preempted by higher priority users. In this case a vendor may supply the 
user with a priority-based probability of access. For example, a high priority is guaranteed 98% 
across the board availability, where a low priority user is guaranteed a 95% off peak probability of 
access and an 80% peak probability. The user should expect compensation should the vendor fail 
to meet these contractual guarantees. It is highly probable the naval forces would desire 
(purchase) the highest degree of service and then control access to individual users by controlling 
access to the SATCOM terminal. 

b) Delay and Blockage 

Untimely delivery and failure to deliver information are clearly critical quality 
concerns. The user desires a consistent level of sen/ice and therefore desires to know how 
reliable the system is in terms of message delivery. When evaluating blockage and delay the 
assumption is made that the system is completely available and the user has access. Delay and 
blockage are measured as the probability of no delay P(no delay), and the probability of no 

For example, if a commercial wideband system was included as part of ADNS on board a ship 
(i.e., a commercial CAP written), then the ADNS priority scheme could control the shipboard 
usage of the high priority line. 
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blockage, P(no blockage) Again the user desires consistent service regardless of the network 
congestion level and therefore these values should be associated with advertised level of service 
(i.e., high priority user versus low priority user). If the vendor does not provide a priority-based 
service then worst case maximum network congestion should be used to model and predict delay 
and blockage values. This will obviously decrease a vendor’s delay and blockage performance but 
naval users can not afford (in dollars or lives) delayed or blocked messages during a crisis. 
Furthermore, the penalty for not providing a priority-based service should create an incentive to do 
so. In the author’s opinion priority based service is not a uniquely naval need, critical financial 
networks or powerful news organizations desire the same capability. 

c) Robustness 

The robustness performance measure reflects a system’s ability to degrade 
gracefully and then restore service gracefully. Naval force users continually train to handle 
unexpected failures and then effectively and efficiently recover from those failures. If is certainly 
recognized that anomalies can cause significant service degradations. These anomalies can be 
due to an unexpected hardware or software failure, sabotage, poor network management or a 
host of other causes. The robustness performance measure doesn’t attempt to predict when and 
how degradation will occur, instead the goal is to identify vendors that possess the ability to 
efficiently and effectively handle the problem. As a customer DoD will not be able to dictate how a 
vendor trains their employees or what procedures the vendor uses while handling a network 
emergency. Instead DoD and therefore the Navy and Marine Corps will rely on contractual 
agreements that guarantee through monetary incentives (i.e., user fees and vendor rebates) 
consistent levels of performance during system anomalies. It is clearly understood that unless 
some degree of negligence can be proven the vendor should not be held accountable for failure to 

20 The inverse P(delay) and P(blockage) is not used because it conflicts with the human tendency 
towards the notion of "bigger is better." Objective values of P(delay) and P(blockage) would be 
very small values (i.e., <2%), whereas objective values for P(no delay) and P(no blockage) large 
numbers (i.e., >98%).The bottom line, humans associate bigger with better. 
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adhere to the contract given some magnanimous natural disaster^"* However the fact remains, 
bad things happen and service will be degraded. 

When these emergencies occur naval users want to know with some degree of 
certainty what level of service they can rely on. As previously presented a customer priority 
scheme is a way to provide a level of availability, delay and blockage assurance. During a crisis 
where degradation occurs high priority customers should expect (within some contractual 
tolerances) a constant level of service. This constant level of service would be achieved by 
rerouting remaining capabilities away from low priority customers to high priority customers. 
Furthermore, as full capacity service is restored the high priority customer would receive his full 
capabilities prior to lower tiered customers. High priority customers could incur a monetary 
premium for this type of service, or if the market is competitive a large customer (i.e., DoD) may 
be able to negotiate for service priority in return for an exclusive use contract. 

It is within this background and from this line of reasoning that within the 
proposed hierarchy the robustness MOP is considered to be binary. That is the service provider 
either has a customer priority scheme of he does not. In a time of crisis naval forces do not 
consider themselves equal to other commercial customers. Marines and sailors lives may 
depended on critical information links and those links must be there. Lack of availability, and delay 
or blockage of service because of head to head (i.e., unprioritized service) competition with 
civilian users is unacceptable to the naval user. Historical precedent for this need is easily cited. 
For example, when a natural disaster such as an earthquake occurs in a major U.S. city 
telephone networks are often degraded. The remaining networks become extremely congested by 
users attempting to collect or report information about the disaster. When this occurs telephone 
providers in cooperation with state and local governments attempt to restrict usage to identified 
“essential users." In the case of a natural disaster essential users are police, fire and medical 
services. The same idea is exactly transferable to a military crisis where naval forces consider 

For example, a critical number of satellites are destroyed during an asteroid storm, or a major 
earthquake destroys a critical network operations center. 
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themselves to be essential users and therefore demand highest priority during a service 
degradation. It should be obvious that politics associated with an international consortium may 
impact a provider’s willingness to allow the U.S. military to be a high priority user. Measurement of 
that political element is reflected in the access and control MOE. The goal of the robustness MOP 
is to identify if the option for priority service even exists. Lastly, this priority service should require 
little manual effort to initiate. That is priority service should be an automated process that can 
intervene the instant service degradation occurs. In summary the robustness MOP reflects the 
existence of an automated customer priority scheme. 

d) Data Integrity 

The integrity of exchanged information is of critical importance to the naval user. 
Inaccurate, ambiguous and corrupted data can dramatically reduce combat effectiveness. Bit 
error rate (BER) is the primary means for measuring data accuracy. As presented in the 
discussion of the coverage MOE (refer to Table 4, page 38) acceptable BER is determined by the 
applications that the user is running. The evaluation of coverage and capacity relied on the use of 
a standard terminal operating at a fixed BER. The data integrity MOP is used to identify a 
system’s BER and that value can be used in coverage and capacity evaluations. Simply put a 
user’s faith and therefore reliance on a system reduces if BER falls below a required value. Loss 
of faith in an IT system (i.e.. Does that message really say drop the bombs here?) constrains 
naval force power project capabilities. 

e) Weather 

Due to their expeditionary nature and the push to operate in littoral regions naval 
forces execute their mission in a variety of weather conditions. The intent of the weather MOP is 
to reflect a system’s capability to provide service in bad weather. Presentation of some 
background information is necessary prior to defining how the weather MOP is used within the 
propose hierarchy. 
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Attenuation of radio waves is not only caused by free-space loss but particulate 
contributions due to rain, fog, mist, and haze (dust, smoke, and salt particles in the air) as well. 
[Freeman, 1991, pp. 494] Atmospheric effects can have a significant impact on radio frequencies 
above 10GHz were wavelength is close to the size of atmospheric particulate. Often in link margin 
calculations free-space loss is the dominant attenuation factor and loss due to particulate is 
comparably small and therefore ignored. Furthermore, concentration of particulate matter is 
difficult to predict and model and therefore removal of its effects for baseline calculations is 
acceptable. Given a baseline, communications engineers can seek ways to introduce link margin 
to "cover" attenuation caused by some pre-determined constant level of atmospheric particulate. 
The communications engineer’s goal is to provide enough margin to ensure a pre-determined 
path availability (usually measured as a probability, i.e., P(available path)). 

As indicated particulate attenuation can be negligible for most cases (i.e., fog). 
However, attenuation due to excessive rainfall can be a significant problem. It is important to 
realize that use of annual rainfall rates provides a poor indication of path availability. Freeman 
provides a proper justification of this position when he states. 



For instance, several weeks of light drizzle will affect the overall long-term path 
availability much less than several good downpours that are short lived (i.e., 20- 
min duration). It is simply this downpour activity for which we need statistics. Such 
downpours are cellular in nature. How big are the cells? What is the rainfall rate in 
the cell? What are the size of the drops and their distribution?[Freeman, 1991, 
pp. 497] 



Using this justification as a defense Freeman goes on to indicate that point 
rainfall rates (the rate of rain as measure over a much smaller range of time i.e., millimeters per 
hour) is a more valuable measure for indicating path availability than annual rain rate.^^ Loss of a 



22 Freeman cites the use of high-speed range gauges that output to computers for minute by 
minute analysis. Further he uses a comparison of rain rates of Florida and Oregon. Oregon has 
the highest annual rainfall within the U.S. (i.e. it rains a lot), while Florida (Miami) has a short-term 
rainfall rate 20 times greater (i.e., when it rains it rains fast) than Oregon. Freeman demonstrates 
that while Oregon gets more annual rain average path availability for a 48GHz signal (Ka band) is 
less in Florida due to the higher short-term rain rate. [Freeman, 1991, pp. 498] 



95 



signal due to rain, or rain fade as it is often called, can be reduced using a variety of techniques. 
Things such as "increasing transmitter po\wer, improving receiver noise factor, increasing antenna 
size,... or ensuring path diversity" are valid techniques to reduce rain fade. [Freeman, 1991, pp. 
499] Path diversity implies that multiple paths between the transmitter and the receiver are 
available (i.e., multiple satellites in view). For LEO wideband systems automatic path switching 
performed by the user terminal should already be included in every terminal due to the need to 
continually switch between satellites during normal operations. 

The goal of the weather MOP is not to dictate how a vendor solves the rain fade 
problem (i.e., how much link margin is provided). Instead the intent is to represent the level of 
service that can be provided during periods of excess rainfall. Therefore the user's desire is to 
measure weather in terms of path availability, reflected as probability of an available path, 
P(available path). That is under what conditions does the service quality degrade to the point at 
which the path is no longer available? For this MOP it is assumed that the user is operating a 
standard terminal as originally defined in the discussion of the coverage MOE. The use of a 
standard terminal not only fixes antenna size and receiver noise factor but it also defines an 
acceptable BER. In the user’s terms a path is not available if the BER falls below that established 
in the definition of the standard terminal. A detailed discussion of how P(available path) is 
calculated is beyond the scope of this thesis and the reader is referred to the Freeman text, pages 
497 to 539, which articulates the relationship between excessive rainfall and path availability. It is 
enough to say that values for worldwide rain rate (rain intensity) are used to predict the specific 
attenuation to a given frequency, which in turn is then related to path availability. [Freeman, 1991, 
pp. 505] Critical values for P(available path) are presented in the Chapter IV discussion of the 
weather MOP utility curve. 

4. Summary 

There are six proposed QoS MOPs each of which relate to the user's desire for assured, 
accurate, unambiguous information exchange. Robustness reflects a system's ability to provide 
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priority service. Availability measures a user’s probability of access. Delay and blockage are both 
recorded as probabilities and reflect the probability of no delays and no blockages. Data integrity 
indicates a systems ability to provide uncorrupted data and the weather MOP provides a measure 
of path availability. A graphical representation of the quality of service MOE is provided in Figure 
37. 



Quality of Service 



Robustness Avsiability Blockage Delay Data Integrity Weather 



Figure 37. Quality of Service MOPs 



K. FLEXIBILITY 

1 . Defined 

The definition for flexibility used in this evaluation hierarchy is identical to that presented in 
the CRD, (see Figure 38). The goal of the flexibility MOE is to identify key logistics requirements 
that impact a system, specifically terminal design. Terminals that are difficult to use or those that 
require extensive maintenance, negate potential benefits the system can bring to the battlefield. A 
system that has poor reliability and is difficult to maintain is a logistical drain. 

Flexibility 

The ability to support the full dynamic range of 
military operations, missions, and environments. 

Figure 38. Flexibility Defined [CRD, 1998, pp. 4-41] 
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2 . 



Why is it Required? 



As show on in Table 10 the CRD provides adequate justification for flexibility 
requirements. As expected naval force flexibility requirements are nearly identical to those at the 
DoD level. The bottom line is that the flexibility is intended to reflect a system’s operational 
suitability. That is, as the flexibility definition states the system must support the full range of naval 
force missions. 



CATEGORY 



Flexibility 



RATIONAL (WHY REQUIRED) 
DoD/JOINT REQUIREMENTS 

• Warfighters prosecute military • 

operations across a wide 
spectrum of conflict. 

• Warfighters need to 

accommodate evolving doctrine, • 

requirements, threats, 
technologies. 

• Emphasis is on fast-paced mobile 

operations. • 

• A wide variety of operating 
frequencies is required to support 
the warfighters’ needs. 

• Warfighters’ must make efficient • 

use of limit frequency spectrum. 

• Systems should be reliable, easy • 

to use, and safe to operate. 



RATIONAL (WHY REQUIRED) 
NAVAL REQUIREMENTS 

Expeditionary nature naval 
warfighters prosecute military 
operations across a wide spectrum 
of conflict in all regions of the world. 
Structured to project power implies 
the need to operate in an 
environment of fast-paced mobile 
operations. 

Self-supporting implies high 
reliability, ease of maintenance, 
adequate user training and 
documentation. 

Unrestricted by poor reliability that 
reduces operational availability. 
Scalable implies the ability to 
leverage and include new 
technologies quickly and efficiently. 



Table 10. Flexibility Justification 
3. Measuring Flexibility 

It is the author’s opinion that the flexibility MOE should reflect the eleven traditional 
logistics elements that measure operational suitability: "operational availability, reliability, 
maintainability, interoperability, compatibility, logistics supportability, transportability, 
documentation, manpower supportability, training requirements, safety and human factors" as well 
as an additional element called upgradability. [Hoivik, 1998] However it is recognized that twelve 
fully characterized MOPs (i.e., a utility curve for each, and AHP weighting of eleven items) does 
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not comply with the original intent of this evaluation hierarchy. That is, the goal is to provide a 
useable evaluation tool that does not severely encumber the user. For this reason the author 
selected what he felt to be the three most important logistics concerns as flexibility MOPs. One 
logistics element, interoperability, has already been addressed within the hierarchy as its own 
MOE. The remaining logistics elements while they are discussed, they are not fully characterized 
as MOPs within this hierarchy. At the very least each of the seven remaining logistics elements 
should be considered as potential MOPs, even if they are included as binary measures. Table 1 1 
provides the results of this breakdown and a discussion of each of the elements follows. 



Use 


Element 


Flexibility MOP 


Operational Availability 


Flexibility MOP 


Reliability 


Flexibility MOP 


Maintainability 


Discussed 


Compatibility 


Discussed 


Logistics Supportability 


Discussed 


Transportability 


Discussed 


Documentation 


Discussed 


Manpower Supportability 


Discussed 


Training Requirements 


Discussed 


Safety and Human Factors 


Discussed 


Upgradability 


Its own MOE 


Interoperability 



Table 11. Logistics Elements 
a) Operational Availability 

As reflected in a Defense Acquisition University (DAU) approved Test and 
Evaluation (T&E) course taught at the Naval Postgraduate School (NPS), operational availability 
(Aq) is; “a measure of the degree to which an item is in an operable and comittable state at the 
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start of a mission when the mission is called for at a random time." [Hoivik, 1998] Unlike the QoS 
availability parameter, Aq is intended to reflect the availability of the user defined standard 
terminal. For this reason Aq calculated as shown in Figure 39. 

Ao = [Operating Time + Standby Time] divided by 

[Operating Time + Standby Time + Total Corrective 
Maint. Time + Total Preventive Maint. Time + Total 
Admin Logistics Downtime] 

Figure 39. Calculation of Operational Availability [Hoivik, 1998] 

Obviously many design and production parameters can influence Aq. The goal is 
to indicate how these availability design parameters impact mission effectiveness. To do this Aq 
values are often segregated into three categories, full mission capable (FMC), partial mission 
capable (PMC), and not mission capable (NMC). The boundary values for each of these 
categories are user determined and should be defined prior to testing. The desire is to have a 
system that has a FMC Aq. Anything other than a FMC system has little value to the naval 
warfighter, therefore within this hierarchy Aq is a binary MOP, (i.e., FMC or not FMC). 

Ideally Aq is evaluated during some form of vendor sponsored operational testing. 
However as mentioned in other sections of this research due to the immature nature of the 
systems involved Ag will probably be obtained through simulation. Evaluators must either identify a 
simulation for vendor’s to use or subjectively critique the vendor created simulation to identify any 
potential biases. The remaining alternative is for the evaluators to create and run their own 
simulations. Regardless of the method used evaluators must know and understand the model’s 
strengths, weakness as well as the assumptions made during its creation. Additionally simulation 
duration should be long enough to reveal availability issues that may not be reflected in short term 
simulation. Simulation duration should reflect standing operational timelines. For example, the 
standard naval force deployment is six months with at least two months of pre-deployment work- 
up. Lastly the simulation must also reflect the intended operational environment. That is 
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temperature range, temperature rate of change, water vapor content (humidity), wind speed, 
various form of precipitation all must be specified by the user. 

b) Reliability 

Reliability is defined as the "duration of probability of failure-fee performance 
under stated conditions.” [Hoivik, 1998] Fault free performance includes both software and 
hardware performance. There is an assortment of accepted methods for measuring reliability and 
in detailed reliability testing numerous measures are used to reflect all elements of reliability. 
However, within this hierarchy the author has chosen to reflect reliability by a single measure. 
Therefore, within this hierarchy the author proposes that “Reliability = number of hours without a 
critical failure, under specified mission conditions." [Hoivik, 1998] Just as with operational 
availability specified operating conditions implies that the user defines a standard terminal and 
establishes guidelines for a reliability simulation. Simulation output should quantify reliability in 
terms of projected mean time between operational availability failure (MTBOMF). MTBOMF 
should be reflected in terms of hours and critical values should reflect desired user reliability 
demands. The flexibility section of Chapter IV discusses this point further. 

c) Maintainability 

Maintainability is defined as "the ability of an item to be retained in or restored to 
specified condition when maintenance is performed by personnel having specified skill levels, 
using prescribed procedures and resources, at each prescribed level of maintenance and repair." 
[Hoivik, 1998] Accurate evaluation of maintainability should include hands on repair by qualified 
personnel, however with a "paper concept" this is not possible. Therefore, evaluators that do not 
have a standard terminal, or a standard terminal mock up must attempt to evaluate maintainability 
by making analogies to existing systems and looking for maintenance friendly characteristics in 
the engineering drawings. Evaluators, should concentrate on trying to quantify the number of 
maintenance events a terminal may require, how time intensive terminal repairs will be, and at 
what level those repairs must be made. Vendor’s should be able to provide written descriptions of 
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projected maintenance concepts and maintainer skill levels. Things such as use of built-in test 
(BIT) and quick and ready access to line replace units (LRU) are things that should be identified. 
From these descriptions evaluators using the input from qualified naval maintainers should 
categorize systems according to their potential maintainability. Within this hierarchy it is proposed 
that system maintainability (since it is being subjectively evaluated from engineering drawings and 
marketing reports) be measured in terms of a maintainability spectrum. The idea is analogous to 
the use of a relationship and legal spectrum under the access and control MOE. The spectrum 
presented in Figure 40 is not intended to be all inclusive, instead readers should use the spectrum 
as a point of departure for refinement of maintainability critical values. 



Poor 

Maintainability 



Drop-dead 

• No use of BIT. 

• Operator level 
maintenance requires 
extensive training 
and effort. 

• Routine maintenance 
impedes operational 
availability. 

• Excessive false 
failure alarm rate. 



<- 



Acceptable 

Maintainability 






Threshold 




Objective 



i 

I 

Use of BIT that fault j 

isolates to LRU using j 

additional external 
test equipment. I 

Operator level | 

maintenance 
exceeds basic care, 
cleaning, and 
replacement of 
expendable. 

Routine maintenance 
exceeds 1 hr per 48 
hr period. 



• Use of BIT that fault 
Isolates to LRU using 
no additional test 
equipment other than 
the terminal itself. 

• Operator level 
maintenance limited 
to basic care, 
cleaning, and 
replacement of 
expendable. 

• Routine maintenance 
shall not exceed 1 hr 
per 48 hr period. 



Figure 40. Maintainability Spectrum 
d) Compatibility 

Compatibility is a parameter that is especially important aboard a "frequency 
congested" Navy ship. A widely accepted definition of compatibility is "the ability of two or more 
items or components of equipment or material to exist or function in the same system or 
environment without mutual interference." [Hoivik, 1998] Traditionally compatibility extends to both 
hardware and software. For the proposed hierarchy the software portion of compatibility is 
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covered under the interoperability MOE, while the hardware aspect of compatibility is address 
here. 

Identification of the operating environment is a critical user input to this definition. 
Within the naval force context operating environment must extend beyond systems that are 
physically collocated on the same platform (i.e., on the same ship) but to any combat system that 
can is "electronically" collocated with the SATCOM terminal. Electronically collocated implies the 
entire region in which an earth terminal’s transmitter power can influence other combat systems. 
For example, while operating in a standard formation earth terminals from one ship should not 
impede the operation of another ship’s radar or communications systems. Ideally the vendor’s 
earth terminal should not force users to modify warfighting tactics, (i.e., force a CG to modify the 
operating pattern of its primary air search radar). Lack of compatibility must not be allowed to 
impact force operational effectiveness. 

. e) Logistics Supportability 

Logistics supportability encompasses a wide range of topics however with in the 
context of this discussion the author present logistics supportability in terms of supply support. 
Future users of this hierarchy must be cognizant of how repair parts will be supplied and how 
depot level maintenance will be performed. In recent times significant focus has been given to 
"outsourcing" both of these aspects of logistics support. The intent of this discussion is not to 
sway the reader either way however the desire is to present factors that should be considered 
when a system is evaluated. 

Given that this thesis focuses on naval use of commercial SATCOM it is 
conceivable that support service could be procured with the purchase (or lease) of a large number 
of terminals. Many contractual arrangements can be made in terms of what level of logistics 
supportability will be provided. For example, for corrective level maintenance above the operator 
level the Navy and Marine Corps could enter into a maintenance float concept with the vendor. In 
this case faulty terminals are repaired by the vendor (i.e., vendor depot) and the user immediately 
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receives a reconditioned (or new/) item from an exiting (military controlled) stock. The critical 
elements in this concept are; the terminal itself is an LRU, the vendor has a robust production and 
repair capability, the military terminal is analogous to the civilian market terminal. Of course if the 
military requires a highly specialized user terminal then military intermediate and depot level 
maintenance may be more appropriate. Regardless system evaluators must consider these 
points. 

In terms of supply support military users of a commercially available product may 
look to having supply support partially performed by the vendor. For example, a contractual 
requirement for the vendor to supply directly to a user located in a combat zone w/ould not be 
operationally effective. Instead an agreement to have the contractor transport repair parts to a 
rear area logistics support area (i.e., transport to mainland Japan for troops engaged in Korea) 
would be valuable. 

f) Transportability 

This logistics element has already been addressed within the hierarchy through 
the use of a standard terminal. A quick survey of most of the emerging commercial wideband 
terminals reveals that they are intended as desk top systems for the average home user. 
Furthermore it is projected that in most cases military users would seek to integrate the terminal 
into different weapon systems (i.e., HMMWV) and therefore the equipment is inherently 
transportable. As long as the systems being evaluated fits into this general category then 
transportability should not be an issue. However, if competing systems either provide a unique 
solution that greatly enhances transportability and mobility then this should be reflected in the 
evaluation. The same idea holds for the reverse situation in which the vendor’s equipment 
impedes for increase transportation requirements. 
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g) Documentation, Manpower Supportability, Training Requirements, 
Safety and Human Factors 

Documentation, manpower, training and safety and human factors are all items 
that should be reviewed for inclusion as potential measures of performance during future 
evaluations. Evaluators could also qualitatively evaluate a vendor’s ability in each of these areas 
and include them in a subjective measure during source selection. 

In terms of documentation evaluators should seek to identify how documentation 
is provided and how often it is updated. Ideally documentation should be provided electronically in 
a manner that allows for easy inclusion of changes. Manpower supportability requirements should 
not force naval users to create new military occupational specialties or dramatically increase the 
required numbers of specific specialties.^^ Ideally a vendor would supply a system that at a 
minimum would be manpower neutral (i.e., no increase in manpower required) at a maximum the 
system would allow for a decrease in manpower requirements. 

Closely coupled to manpower, are training requirements. Ideally a user terminal 
should be a plug and play system that a computer literate user can operate with minimal training. 
Training should be provided to maintainers and operators in a variety of formats, (i.e., classroom, 
video, written manuals). It is likely that use of a commercial system will require little training since 
it will be marketed to the civilian home user. 

Safety and human factors issues are best represented by their individual 
definitions. Safety is defined as "freedom from those conditions that can cause death, injury, 
occupational illness, damage to or loss of equipment or property, or damage to the environment." 
[Hoivik, 1998] It is anticipated that as a desktop computer like system a SATCOM earth terminal 
(unless it is transmitting at exceptionally high power) provides a negligible hazard to naval 
operators. Furthermore, given the commercial nature of the product it is unlikely that 
environmentally hazardous materials (except possibly batteries) will be included in the product. 

23 In general communications and computer equipment is operated and maintained by: USMC 
operators MOS 2500, USMC maintainers MOS2800, USN operators RMs, USN maintainers ETs. 
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Human factors "are those elements of system operation and maintenance which influence the 
efficiency with which people can use systems to accomplish the operational mission (man- 
machine interface)." [Hoivik, 1 998] Within the context of the proposed hierarchy future evaluators 
may consider man-machine interface factors such as software interface for both the operator and 
maintenance personnel and ease of use in terms of operating in a combat environment (i.e., is 
there a 15 step process to turn the system on?). 

h) Upgradability 

The CRD accurately characterizes upgradability as the capacity to "facilitate rapid 
and orderly enhancements and upgrades to operational capabilities and features." [CRD, 1998, 
pp. 4-43] The fundamental premise behind this need is that warfighter requirements will continue 
to grow and evolve and supporting SATCOM systems should have the capacity to "leverage 
advanced technologies and new commercial offerings" in an efficient and timely manner. ." [CRD, 
1998, pp. 4-43] During the evaluation process users may desire to reward or penalize vendor’s for 
their attempt to or failure to provide open systems that enhance upgradability. 

4. Summary 

In summary it was the author’s decision to limit the flexibility MOE to three critical logistics 
elements (refer to Figure 41). Although not include in the hierarchy as fully characterized MOPS a 
discussion of seven other logistics elements is provided for evaluation by future users as potential 
MOPs. Potential drop-dead, threshold and objective values for the three MOPs are proposed in 
the flexibility section of Chapter IV. 
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Rexibility 



Operational A\railability Reliability Maintainability 

Other factors to consider Compatibility, Logistics Supportability, Transportability, Documentation, Manpower 
Supporiabilrty, Training Requirements, Safety & Human Factors, Upgradability, 

Figure 41. Flexibility MOPs 

L. SUMMARY 

In summary seven MOEs are presented and each was derived from the JROC approved 
CRD. In some cases the MOPs that represent the MOEs are identical to those issues discussed 
in the CRD. However in some instances new MOEs that better reflect the issues unique to naval 
use of commercial SATCOM were created and defined. Figure 42 graphically represents the full 
evaluation hierarchy. 
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IV. 



WEIGHTING AND UTILITY ANALYSIS 



As discussed in Chapter III in order to effectively evaluate competing concepts MOEs and 
MOPs must be weighted and summed to provide an overall system effectiveness value (SEV). 
Prior to weighting, the critical values for each performance measure and the user’s utility, or the 
value that the user has for a particular capability, must be determined. In order to provide an SEV 
that discriminates between each system’s ability to meet user needs, utility curves are needed to 
reflect the marginal value that a user has for increasing (decreasing) levels of performance. 
Discussion of weights and identification of critical values are the focus of this chapter. 

The final output of this chapter is "strawman" evaluation hierarchy. The methodology used 
to construct the “strawman" hierarchy begins with a presentation of utility analysis to include 
determination of dropdead, threshold and objective performance values. As the chapter 
progresses discussions of MOP and MOE weighting are provided, however final weights are not 
assigned. Instead an ordinal ranking is generated giving the user a point of departure for team 
determination of MOP and MOE weights. It is the author’s opinion that performance measure 
weighting is highly situational dependent and should be performed by a group of concerned users 
(i.e., warfighter representatives from each concerned community). Therefore determination of 
relative weights is beyond the scope of this thesis^^. Additionally it is recognized that critical utility 
values (dropdead, threshold, and objective) are also situational dependent. However creation of a 
strawman hierarchy inherently implies that these values are subject to and should be changed as 
a concerned team deems appropriate. The primary emphasis of this chapter is to present a 
discussion of the topics relevant to each performance measure so that future users have a forum 
from which to begin construction of a specific wideband evaluation hierarchy. The ultimate goal is 
to have a future naval commercial SATCOM integrated product team evaluate competing 
wideband systems using a hierarchy derived from the topics presented in this thesis. 

24 It is the author’s intent to present potential weighting schemes that reflect solicited user input in 
a separate follow-on report. 
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A. 



UTILITY METHODOLOGY 



As a reminder, each MOP measures the degree to which a system satisfies a specific 
user need by representing that need as a system design parameter. Taken together MOPs for a 
given functional objective reflect a system’s ability to accomplish that functional objective. 
Functional objectives are reflected numerically as MOEs. Collectively the seven functional 
objectives (MOEs) provide a measure of overall system performance and are recorded 
numerically as a SEV. In order to effectively discriminate between the capabilities of potential 
solutions user defined critical values for each performance measure. A utility function is used to 
map the range of acceptable values of a MOP to a [0,1] scale. This mapping reflects the 
incremental value that the user group has for increases/decreases in capability. 

Construction of a utility function must begin at some point, and a traditional method is to 
identify three critical poipts of performance. These three points are: 

• Dropdead: the level of performance below which the system has no value and should not 
be used 

• Threshold: the minimum acceptable performance 

• Objective: the ideal, desired level of performance. [MILSATCOM, 1993, pp. 5] 

Once identified the points can be assigned generic starting utility values; dropdead = 0.0, 
threshold = 0.5, and objective = 1.0. From there an initial assignment of a linear function can be 
used to represent the user’s reward curve. Linear utility implies that an incremental increase in 
performance is directly proportional to a user’s incremental increase in value. Often however, the 
user’s preference for increased combat effectiveness is not linear and therefore linear utility is 
usually not representative of the real world. The user’s utility curve should reflect this non-linear 
preference. Figure 43 and Figure 44 graphically represent the issues presented above and are 
further discussed in the following paragraphs. 
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1.0 




Figure 43. Linear and Non-linear Utility Curves 

Figure 43 represents three different preferences for value. That is, the curves answer the 

question; how much value does a user have for a specific level of performance? The top curve in 
Figure 43 represents a non-linear utility curve modeled using User Utility = (Level of Performance) 
Initially for this curve, the incremental change in value is greater than the incremental change 
in performance therefore the "payoff" for a small increase in performance is high. However once 
past the threshold, as performance approaches the objective level, utility increases at a 
decreasing rate. 

The bottom curve is represented using User Utility = (Level of Performance) and 
indicates the user’s preference for attaining objective performance by assigning threshold 
performance a relatively low utility. Furthermore as an incentive for increased performance 
towards the objective level, utility increases at an increasing rate. Therefore, initially each 
incremental change in performance has little value, or low payoff however increased performance 
after threshold has an incrementally higher value. 

The center curve reflects linear utility (in this case slope = 1). This curve implies that the 
user is indifferent. An incremental change in performance is equal to an incremental change in 
value over the entire range of performance values. As previously presented linear utility does not 
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usually reflect real world conditions. In fact none of the three curves completely reflect real world 
situations. As one would assume some combination of the three curves more accurately reflects 
real world scenarios. The non-linear curve presented in Figure 44 is one of the many possible 
ways that these utility functions can be combined. 




Figure 44. Representative Non-Linear Utility Curve 

What is important about the curve in Figure 44 is that it states that the user’s utility varies over the 
range of performance. The user is unwilling to accept below threshold performance as indicated 
by the assignment of low value. Incremental value declines as objective performance is attained 
due to the fact that utility above threshold increases at a decreasing rate. The only element of 
linear utility that exists in this curve is that threshold performance and utility are of equal numerical 
value. Although this is not always the case assignment of 0.5 utility for threshold performance 
does have value in acting as an anchor point for users. Users tend to prefer balance, and like 
categorizing performance as above or below threshold. Assignment of 0.5 utility facilitates this 
user desire by providing a balanced reference point. For example; 



112 



When tasked with conducting an Operational Effectiveness Analysis as part of 
the MILSATCOM Polar Adjunct COEA, Air Force Space Command identified 25 
distinct MOEs. Initially each MOE was assigned a linear utility function with 
dropdead = 0.0, threshold = 0.5 and objective performance = 1 .0. The linear utility 
curves were presented to users who were given the chance to modify them. It 
was found that users did not hesitate to modify the performance values (i.e., 
increase/decrease the range of performance) to reflect operational preferences or 
desired capabilities. However, once the performance range was modified the 
utilities assigned to the end points remained consistent with the original curve 
(i.e., dropdead = 0.0 and objective = 1.0). Furthermore, despite the fact that 
internal points were added to emphasize certain levels of desired performance, 
24 of the 25 MOE threshold values were assigned a utility value equal to 0.5. In 
other words the users did not modify the strawman threshold value of 0.5. 
[Paraphrased from MILSATCOM, 1993, pp. 16] 



What the MILSATCOM Polar Adjunct example demonstrates is that users can balance 
their utility using a threshold value of 0.5. The approach taken in this thesis is identical to that 
executed by the Polar Adjunct team. That is, create strawman utility curves and allow users to 
refine them to reflect operational preferences.^^ 

Lastly in order to accurately articulate user needs and value, MOPs and MOEs must be 
weighted prior to summation. The reader should recall that: 

. MOEi=I(Wj* MOPj) 

• SEV=I(Wi* MOEi) 



The next section discusses the methodology that should be applied to accurately weight 
MOPs and MOEs. 

B. WEIGHTING METHODOLOGY 

Ensuring that all MOPs within a given functional objective are not equally important allows 
the user to indicate preference for different system design capabilities. This preference should 
directly reflect a user’s needs priority. That is, a higher ranked MOP implies that the need it 
represents has higher relative importance than the need represented by a lower ranked MOP. A 

25 As stated earlier collection of user input and subsequent refinement of strawman utility curves 
is beyond the scope of this thesis. 
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priority ranking of MOPs is not enough, in that it does not provide a measure for comparison 
between MOPs. The only information that an ordinal list provides is that number one is better than 
number two. The ranking does not stay how much better number one is than number two. 
Therefore, drawing on the utility theory background previously presented it is important that each 
MOP is weighted relative to each other. That is, what is the relative value of MOP 1 compared to 
MOP 2, 3, 4, 5 etc.? This same argument is also applicable to the weighting of MOEs to ensure 
that each MOE contributes the proper (user determined) amount to the calculation of a SEV. The 
Analytic Hierarchy Process is the recommended method that should be used to perform MOP and 
MOE weighting. 

1. Analytic Hierarchy Process (AH P) 

The Analytic Hierarchy Process (AHP) is a user-friendly decision support technique. 
AHP’s position is that direct assignment of weights by an evaluator or group of evaluators is "too 
abstract and will result in inaccuracies." [MILSATCOM, 1993, pp. 33] Therefore, pair-wise 
comparisons are used to allow the evaluators to reveal their preferences. Through the use of pair- 
wise comparisons AHP provides a method for revealing the relative value of a ranked set of 
performance parameters. 

Fundamental to the use of AHP is the construction of a decision hierarchy analogous to 
the evaluation hierarchy presented in this thesis. Evaluation measures at each level (i.e., MOPs) 
undergo pair-wise comparison, their relative weights are revealed and the measures can then be 
summed for determination of higher level values (i.e., MOEs). The pair-wise comparison 
processes is repeated and the weighted MOEs are summed to produce the SEV. The result is a 
SEV that represents the users’ preferences. 

Pair-wise comparisons can be made numerically or verbally. The user should be allowed 
to determine which method of comparison they prefer. In either case a quantitative score is still 
the final product. The underlying concern is to allow the user to express a definition of relative 
value. Pair-wise comparisons are conducted using a square matrix, where the header row and 
header column are the measures to be compared and the intersecting cells are were the 
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comparisons occurs. A comparison is stated from the perspective of the row. That is, how much 
better is the row value than the column value. 





lAA 


FA 


lAA 


1 


1/2 


FA 


2 


1 



Table 12. Example AHP Matrix Using Numercial Comparison 

For example, the numerical comparison takes the form: pick the number X such that 
MOP#1 is X times more important than MOP #2. Table 12, provides an example using two of the 
four coverage MOPs. Realistically all four MOPs would be included in the matrix however the use 
of two MOPs is used to illustrate the pair-wise comparison technique. The value 2, in row 2 
column 1, states that FA is two times more important than lAA. The opposite comparison, which is 
lAA compared to FA, should be the reciprocal of the FA to lAA comparison, which in this case is 
1/2. The desire to automatically assign the reciprocal to the reverse comparison should be 
suppressed. User response to the opposite comparison is important in that it provides a method 
for determining user consistency. Consistency is discussed in more detail in the last paragraphs 
of this section. A value of 1 indicates that the user is neutral and has no preference between 
either of the compared MOPs. The diagonal of the comparison matrix is always 1 indicating that 
when compared to itself, each element has equal importance. [Zahedi, 1986, pp. 98] 

The verbal comparison consists of words that indicate levels of relative preference such 
as: absolutely preferred, strongly preferred, moderately preferred, weakly preferred, or equally 
preferred. Therefore each cell in the matrix would contain the word that best describes the users 
interpretation of relative value. Once a verbal comparison is made the words are converted to 
numerical values. Equivalent values for the words presented above could be: 

• Absolutely preferred = 9 

• Strongly preferred = 7 

• Moderately preferred = 5 

• Weakly preferred = 3 
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Equally preferred = 1 



From this point AHP takes the pair-wise comparison matrices as input and produces the 
relative weights of elements at each level as output. Although a rigorous proof and presentation of 
the refinement of weights is not appropriate for this thesis a scaled down presentation of the 
method follows. 

Given the following example: [Render, 1997pp. 563-564] 





MOP 1 


MOP 2 


MOP 3 


MOP 1 


Equal 


Strongly 

preferred 


Moderately 

preferred 


MOP 2 


- 


Equal 


Weakly 

preferred 


MOP 3 


- 


- 


Equal 



Table 13. Sample AHP Matrix Using Verbal Comparison 

An evaluator verbally conducts the pair-wise comparison and indicates his preferences for 
three MOPs as presented in Table 13. This evaluation is converted to numerical values and the 
individual columns are summed (refer to Table 14). 





MOP 1 


MOP 2 


MOP 3 


MOP 1 


1 


7 


5 


MOP 2 


.14 


1 


' 3 


MOP 3 


.20 


.33 


1 


Column 


1.34 


8.33 


8 


totals 









Table 14. Conversion to Numerical Value and Calculation of Column Totals 
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MOP 1 


MOP 2 


MOP 3 


MOP 1 


0.746 


0.840 


0.625 


MOP 2 


0.104 


0.120 


0.375 


MOP 3 


0.149 


0.040 


0.125 



Table 15. Cell Values Divided by Column Total 



A new matrix is created wherein each cell is the result of the cell value divided by the 
column total. For example, as shown in Table 15, cell (1,1) is equal to 0.746, which is 1 divided by 
1.34. Weights are then determined by averaging the row values. For this example resulting 
weights are: 

• MOP 1 = 0.737 = (0.746 + 0.840 + 0.625)/3 

• MOP 2 = 0.200 = (0.104 + 0.120 + 0.375)/3 

• MOP 3 = 0.105 = (0.149 + 0.040 + 0.125)/3 







MOP #1 = 70% 




> 



MOP #2 = 20% 



< 



MOP #3 = 10% 






MOE = (. 7*MOP*1) + (.2'M0P#2) + (. 1 ‘MOP#3) 






Figure 45. Bar Graph Representation of Weighting and Calculation of MOE 



Figure 45 provides a graphical representation of the resultant weighting. This procedure 
can be used for n number of MOPs/MOEs however user’s tend to become saturated when 
performing pair-wise comparisons for more than nine measures. [Zahedi, 1986] 
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A downfall of AHP is that the user can be inconsistent when conducting pair-wise 
comparisons. For example, if a user states that A is two times more important than B, then the 
user should also state that B is half as important as A. Inconsistency leads to faulty and 
ambiguous weighting. A consistency ratio can be calculated and used to indicate a user’s level of 
consistency. A detailed derivation of consistency ratio calculation is not required for this thesis. 
The reader is referred to Render or Zahedi for further clarification. For the purposes of this 
presentation it is enough to say that "in general, if the consistency ration is 0.10 (10%) or less, the 
decision maker’s answers are relatively consistent. For a consistency ratio that is greater than 
0.10, the decision maker should seriously consider reevaluating the pair-wise comparisons." 
[Render, 1997, pp. 565] AHP is concluded once a consistency ratio of 0.10 or less is attained. 

The primary benefit of AHP is that it allows users to become deeply involved in the 
weighting process without having to learn unique processes. Users just have to conduct pair-wise 
comparisons, analysts can prepare the evaluation matrices and execute the calculations. 
Furthermore, AHP can be performed using automated support tools, thereby further reducing the 
level of effort required by the users. 

a) Automated Support Tools 

There are various decision support tools that can be used to automate the 
weighting process. A very brief description of two such tools is provided here. They are. Systems 
Engineering and Evaluation Regime (SEER) and Expert Choice. 

SEER is a decision support tool developed by The Aerospace Corporation for 
Naval Space Command. It is a five stage Microsoft Excel based application that allows a small 
team to execute a "customer focused benefit-cost analysis." [Leake, pp.1] The primary benefit of 
SEER is that it allows the user to enter the process at different levels. The lowest level Stage 1 , 
assists the user in framing the analysis by providing a method for MOP and MOE development. 
Stage 1 is clearly applicable for a group that would like to refine the MOPs and MOEs presented 
in this thesis. An interested team could use SEER to investigate what impact additions or 
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deletions to the proposed list of MOPs and MOEs would have. Furthermore, SEER provides the 
user the ability to make projects about future needs and how those needs factor into the 
evaluation of a system. The second level that a user can enter SEER is at Stage 3. Stage 3 
assumes that the user has framed the analysis and has identified the proper MOPs and MOEs. 
From this point the SEER user progresses into Stage 4 and performs an AHP like pair-wise 
comparison of performance measures, resulting in a set of weighted measures. Competing 
system capabilities are entered and system effectiveness scores are automatically calculated. 
The final stage. Stage 5, allows the user to inject cost considerations into the decision process. 
Stage 5 output is a system benefit-cost ratio. Appendix B, is a paper that fully describes SEER 
and the methodology that it embraces. 

Expert Choice is a software package that was originally developed in 1984 by 
Decision Support Software Inc. [Zahedi, 1986, p. 99]. It is a widely used decision support tool and 
uses an AHP derived algorithm to allow users to interactively weight decision variables. An added 
feature is that Expert Choice executes consistency checks for each of the input matrices thereby 
automating the weight validation process. 

C. SENSITIVITY ANALYSIS 

Critical to a decision maker’s trust of an evaluation hierarchy is an understanding of how 
sensitive that hierarchy is to the weighing schemes and utility functions used. The decision-maker 
must feel confident that weights or utility functions are reflective of their desires and not somehow 
driving at an inappropriate decision. Sensitivity analysis is the manner in which analysts determine 
how sensitive a decision support system is to weighting schemes and utility functions. 

Although sensitivity analysis is not performed on the hierarchy presented in this thesis^® 
the author does recommend that future groups assess their hierarchy in three ways. These 
methods are borrowed from those employed by the MILSATCOM Polar Adjunct evaluation team. 

26 Sensitivity analysis is not performed because user derived utility functions and weighting 
schemes are not presented. 
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[MILSATCOM, 1993, pp. 29] Each of the three methods should be compared to the outcome of 
the original user defined hierarchy. Two of the methods involve manipulation of the weighting 
scheme and the third and easiest to employ is modification of the utility functions. 

Addressing the easiest to accomplish analysis first the analysts should attempt to quantify 
how user defined MOP utility impacts SEV. This can be accomplished by redefining all utility 
curves as linear functions and then recalculating SEV. Does the SEV for specific systems 
change? Does the overall (relative) ranking of competing systems change? At the completion of 
the MILSATCOM Polar Adjunct Operational Effectiveness Analysis, the analysts found that 
relative ranking was effected to a limited extent. [MILSATCOM, 1993, pp. 59] For example, 
although the same alternatives may still be the top four alternatives, the original number three 
ranked system may have become the number two ranked system. This implies that if a quick 
study of the competing systems is needed and user input can not be obtained then linear utility will 
at least highlight the contending systems. This fact could be used to scale down the field if 
numerous systems required evaluation. 

The final two analysis methods modify the weighting schemes. The goal of this process is 
to identify MOP and MOE weights that may "drive" the SEV. The user may have intentionally 
generated this bias to create vendor incentives, or the user may have accidentally created a 
driving weighting scheme. In either case the focus is on identifying the impact that the AHP 
calculated weighting schemes have on the SEV. 

The first modified weighting scheme is the use of uniform weights. The desire is to 
compare original SEVs with those produced by a hierarchy in which all weights are equal. That is 
if there are four MOPs for a given MOE then each MOP contributes 25%. Given that there are 
seven functional objectives (MOEs) each MOE would be equally weighted at 0.143. Comparison 
of uniform weighted SEVs with original SEVs should assist in highlighting weighting biases. 

The second method involves assignment of equal importance to each pair-wise 
comparison. That is uniformly apply the same importance to each compared pair. For example as 
shown in Table 16 each pair-wise comparison is moderately preferred. 
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MOP 1 


MOP 2 


MOP 3 


MOP 1 


Equal 


Moderately 

preferred 


Moderately 

preferred 


MOP 2 


- 


Equal 


Moderately 

preferred 


MOP 3 


- 


- 


Equal 






MOP 1 


MOP 2 


MOP 3 


MOP 1 


1 


3 


3 


MOP 2 


.33 


1 


3 


MOP 3 


.33 


.33 


1 


Column 

total 


1.66 


4.33 


4 





MOP 1 


MOP 2 


MOP 3 


MOP 1 


0.602 


0.693 


0.750 


MOP 2 


0.199 


0.230 


0.750 


MOP 3 


0.199 


0.076 


0.250 



Table 16. Assignment of Equal Importance to Each Pair-wise Comparison 

Comparison of the resulting weights with those presented calculated previously (refer to 
Figure 45) show that the weighting for MOPs 1 and 2 does change. 

• MOP 1 = 0.60 = (0.602 + 0.693 + 0.750)/3 

. MOP 2 = 0.30 = (0.199 + 0.230 + 0.750)/3 

• MOP 3 = 0.10 = (0.199 + 0.076 + 0.250)/3 

• Where previous calculation revealed MOP 1 = 0.737 , MOP 2 = 0.200 and MOP 3 = 0.105 
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The significance of the change becomes evident \when original SEVs are compared to 
those calculated using the equally important weighting scheme. Again the desire is to identify any 
weighting bias that is hidden within the hierarchy. 



D. COVERAGE MEASURES OF PERFORMANCE 

The following sections beginning with this section on coverage present the performance 
measure utility and rankings for the proposed evaluation hierarchy. 

1. Geographic Coverage 

As previously stated linear utility is presented as a strawman for both lAA and FA utility. 
Although utility and the exact performance (i.e., level of lAA or FA desired) may vary the following 
ideas are expressed in the utility curves shown in Figure 46 and Figure 47: 

• South Polar coverage is excluded since there are currently no, too little naval wideband 
SATCOM requirements in that region. 

• Dropdead = 0.0 = Single Theater coverage. 

• Threshold = 0.5 = One Region & CONUS coverage. 

• Objective = 1.0 = North Polar & Worldwide coverage. 

• The selection of One Region & CONUS demonstrates that connectivity between CONUS 
(i.e., warfighter support agencies) and a regional command has more value that 
connectivity between regions. Inter-regional connectivity, other than that to CONUS is 
expressed as above threshold performance. 

• As an example GBS would receive an above threshold lAA score between 0.50 and 0.66 
because it provides for two region and partial CONUS coverage, and a FA score of 0.0 or 
dropdead for Single Theater coverage.^^ 



27 This ranking does not degrade the value of the current GBS system. What it indicates is that 
GBS is the first step towards solving the wideband problem. If GBS was an across the board 
threshold or better system then current plans or talk of GBS Phase III or DoD Wideband Gapfiller 
would not be needed. 
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instantaneous Access 



Figure 46. iAA Utility Curve 



In 

0 . 9 - 




Footprint Area 



Figure 47. FA Utility 

2. Time Coverage 

In the author’s opinion the utility functions of the remaining coverage MOPs TIV and TBV 
are binary; either you have it or you don’t. In almost all cases users desire maximum flexibility and 
therefore tend towards a 24-hour TIV and a 0-hour TBV. Basically the user always wants to be in 
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the FA. To comply with future naval doctrine and visions will be operationally unacceptable to not 
have access to the wideband system. Furthermore, given the fiscally constrained environment 
reliance on other wideband systems (i.e., the luxury of complete back up systems) is not practical 
or probable. Although alternate communications paths will exist trades will have to be made (i.e., 
loss of operational capability) when system reconfigurations are required. Therefore, it is projected 
that the user’s preference is binary. The user’s value for a system that can not provide 24-hour 
time in view, with no gaps in coverage (i.e., TBV= 0-hours) is 0. Is this a realistic position? In the 
author’s opinion it is a valid position. The emerging commercial systems, (i.e., Teledesic and 
Astrolink) are being designed to meet this level of need. The notion that users will be blind, and 
have to wait for a satellite to come into view is unacceptable to the warfighter. Therefore for the 
time coverage MOPs there are only two performance values, objective and drop-dead. 

• Objective = 1.0 when TIV = 24-hours and TBV= 0-hours. 

• Drop-dead = 0.0 when TIV < 24-hours and TBV> 0-hours. 

It is understood that these values are situational dependent however, and significant 
debate about other values should occur. However, from the users standpoint; What good is 
capacity if you can’t use it? The above utility values reflect this user attitude. In summary. Table 17 
list the coverage utility values. 



lAA FA T7V TBV Utility Score 



Drop-dead 


Single Theater 


Single Theater 


< 24-hours 


> 0-hours 


0.0 


Threshold 


One Region & 
CONUS 


One Region & 
CONUS 


- 


- 


0.5 


Objective 


North Polar & 
Worldwide 


North Polar & 
Worldwide 


24-hours 


0-hours 


1.0 



Table 17. Coverage MOP Utility Values 
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3. Coverage Measures of Performance Ranking 

AHP weighting is not performed in this thesis. As previously indicated weighting is highly 
situational and group dependent and therefore is best left for actual execution. However, to 
provide a point of departure for weighting discussions, the author’s suggest MOP ranking along 
with a brief justification is provided in Table 18. Recall that ranking does not provide a value ratio. 
The #1 MOP is the highest ranked MOP but how much more it is preferred over #2 is not 
indicated in the ranking. User preference is only revealed after conducting the AHP weighting. 

MOP rankings are intended to force users to discuss the relevant issues and concern 
themselves with the topic. User agreement with the rankings provided is not the issue. Instead 
disagreement that leads to debate and ultimately to a re-ranked and weighted set of MOPs 
justified by valid user needs is the goal. With this in mind the coverage MOP ranking is provided 
below, (refer to Table 18) with #1 being the most valued MOP. 



Ranking Justification 

1. TIV Coverage without access is useless. Therefore time coverage is more important 

than geographic coverage. T/\/ implies that the user has access while operating 
within a specified FA. 

2. TBV Same as above, except TSl^ implies that the user is denied access while operating 

within a specified FA. 

3. FA More important than lAA because it is the only place within the geographic 

coverage pattern that user can actually use the satellite. 

4. lAA The least important, however very valuable in that it defines a systems geographic 

coverage capability (i.e., where the FA can be placed). 



Table 18. Coverage MOP Ranking 

E. CAPACITY MEASURES OF PERFORMANCE 
1. Data rate 

Transmit and receive data rate utility curves are presented in Figure 48 and Figure 49. 
Each figure indicates drop-dead, threshold and objectives values and implies linear utility. It is the 
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author’s opinion that naval user’s preference for increased data rate is non-linear. More than likely 
a user’s data rate utility, past threshold, increases at an increasing rate. In short, the user favors 
data rate increases. It should be remembered that the transmit and receive sub-attributes are 
intended to reflect the capabilities of a single terminal. Given the wide range of platforms on which 
a terminal could be placed (i.e., aircraft, advanced amphibious assault vehicle, ship) it is clearly 
possible that some users will require a terminal’s full capacity and others will not use a terminal to 
its fullest potential. Therefore it is possible that some platforms will require multiple terminals or a 
non-standard larger capacity terminal. If multiple small volume antennas are required it is possible 
that they could be networked to numerous users aboard the ship.28 However given the space 
restrictions on most naval platforms it is unlikely that large numbers of antennas would be 
practical. Despite the wide range of potential configurations the user desires to decrease logistics 
and support costs by purchasing a common standard issue terminal for all users. 

Table 19 lists the key utility values for each sub-attribute. It should be recognized that 
drop-dead, threshold and objective values are very application dependent. The values presented 
in the figures and tables where chosen to represent some of the factors that should be considered 
in critical value selection. Transmit and receive drop-dead data rates of 64kbps was selected 
solely on the basis that 1) 64kbps is the DoD definition for wideband and 2) planned use of 
INMARSAT-B will provide this capability and therefore any decrease in capacity is of no value. 
Threshold and objective values are estimates and certainly subject to change. The objective 
values were left as greater than 3.0 Mbps to indicate that capacity requirements are unbounded. 
3.0 Mbps was arbitrarily chosen and represents the author’s unsupported prediction that single 
terminal user needs will rapidly progress towards this value. A threshold value of 512kbps was 
arrived at through informal discussions with experienced naval officers and is not grounded by any 



The potential exists for a commercial wideband system to function aboard a naval vessel as a 
sub-component of the Advanced Digital Network System (ADNS). Single or multiple wideband 
antennas could be multiplexed to the ADNS controller. Use of ADNS implies coding of specific 
Channel Access Protocols (CAP) for each commercial system employed. 
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hard evidence. The bottom line is user’s will need to spend considerable time discussing, 
determining and justifying critical capacity values. 



a) Transmit Data Rate 
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Figure 48. Transmit Data Rate Utility Curve 



b) Receive Data Rate 
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Figure 49. Receive Data Rate Utility Curve 
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Transmit 
Data Rate 


Receive 
Data Rate 


Utility Score 


Drop-dead 


64 kbps 


64 kbps 


0.0 


Threshold 


512 kbps 


512 kbps 


0.5 


Objective 


> 3 Mbps 


> 3 Mbps 


1.0 



Table 19 . Data Rate Sub-Attiribute Utility Values 
c) Sub-attribute rankings 

Ranking of the data rate sub-attributes are provided in Table 20, and indicate a 
preference for a higher receive data rate (asymmetric communications). To what degree receive 
data rate is preferred to transmit data rate is the product of a future AHP analysis. 



Ranking Justification 

1. Receive Data Rate Represents asymmetric information need. 

2. Transmit Data Rate A typical user generates less information that he needs. 



Table 20. Data Rate Sub-attribute Ranking 
d) Data Rate Measures of Performance Calculation 

Data Rate 

MOP = (weight ’ RCV) -i- (weight ’ XMIT) 

I ^ 1 

Transmit Data Rate Receive Data Rate 
Figure 50. Data Rate MOP Calculation 

As indicated in Figure 50 the Data Rate MOP is the weighted sum of the sub- 
attributes. AHP should be used to determine the relative weighting of the sub-attributes and then 
each attribute weight is multiplied by the achieved utility score to arrive at a data rate MOP value. 
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2. Mobility 

Recall that the intent of the mobility MOP is to measure a terminal’s ability to 
provide a constant capacity to a mobile user. Furthermore, the terminal platform is assumed to 
remain inside the system’s FA and therefore the design parameter being tested is the antenna’s 
ability to track system satellites regardless of platform motion. Tracking ability is measured in 
terms of maximum attainable velocity, pitch rate, roll rate and yaw rate. Pitch, roll and yaw 
magnitudes are not considered because it is assumed that the terminal remains within the system 
FA (i.e., the terminal does not exceed elevation angle restrictions). Stated in a more direct 
manner, the mobility MOP provides a measure of how stable a platform must be in order to 
provide the user a constant capacity. Stability implies that the platform must remain relatively 
motionless and therefore a platform’s warfighting capability is directly impacted. 

Determination of critical values for the mobility sub-attributes is very difficult. A 
threshold value that is adequate for one platform will not be adequate for another. The difference 
in values is driven by the differences in intended operational use of the platform that the terminal 
is mounted on. For example, suppose a wideband terminal is installed on an air defense variant of 
the Advanced Amphibious Assault Vehicle (AAAV) and is used to provide video from a remotely 
piloted UAV as well as a receive only composite air picture (i.e. CEC or Link 16 like information). 
The AAAV enters the water in Sea State 2, travels to the beach and traverses numerous sand 
dunes until it reaches its objective. During the transit UAV video is being monitored and used to 
identify key terrain features or enemy concentrations and the air picture is used for air defense 
gunner cueing. Throughout the transit the AAAV will change speed, as well as pitch, roll and yaw 
however, loss of the video and air picture should be minimized. It is anticipated that at times the 
AAAV will loose connectivity because the vehicle will leave the FA by exceeding terminal elevation 
angle requirements. However, for the times that the vehicle remains in the FA the desire is to 
articulate the antenna’s ability to provide a fixed level of service through mechanical or electronic 
antenna steering. The critical values required to maintain connectivity are drastically different than 
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those required for an aircraft or ship.29 Therefore specific numerical values for the mobility sub- 
attributes are not presented, instead critical values are characterized by platform type. Further 
analysis must be done to assign numerical values to each platform type. 

For example, the velocity sub-attribute critical values are: drop-dead is the speed 
of an aircraft carrier (CVN); threshold value is the maximum water speed of the AAAV (25 knots); 
and objective is the speed of the F/A-18 E/F. Selection of these platforms are purely for illustrative 
purposes and intended to spark discussion and debate. The goal is to get users focused on what 
impact the lack of antenna tracking has on operational capability. Basically any platform to the 
right of the commercial vendor’s velocity, pitch, roll, yaw values will not be supported throughout 
all ranges of that platform’s operational capabilities. 

Strawman pitch, roll and yaw rate utility curves are also presented using the CVN, 
AAAV and F/A-18E/F as drop-dead, threshold and objective values. Figure 51, Figure 52 and 
Table 21 show the relative utility values for each platform. Pitch, roll and yaw rate are displayed on 
the same curve for ease of presentation and it is recognized that these values could be different. 
In the case of pitch, roll and yaw, utility increases as a terminal’s ability to accommodate less 
stable platforms increases (i.e., a CVN is designed to be stable whereas a tactical aircraft is 
designed to be maneuverable and therefore unstable). Of particular concern is the sea state that 
the sea or amphibious platforms are operating in. The user must identify a sea state and 
understand the impact that the chosen sea state has each platform. For example an AAAV in sea 
state 2 will pitch, roll and yaw significantly more than the same vehicle in sea state 1 . It is critical 
that the users identify, possibly by contacting the different platform program offices, the pitch, roll 
and yaw rates for each vehicle in each operating condition (i.e.. What is the average pitch rate for 
an AAAV in sea state 1?). 



29 Use of wideband SATCOM to tactical aircraft could provide real-time BDA to target planners 
and could increase pilot situational awareness by providing real-time imagery or SIGINT 
information directly to the cockpit. 
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Finally it was intentionally decided that each of the mobility parameters should be 
evaluated in isolation. Although this does not reflect real world conditions in which each of the four 
motions are coupled and occur simultaneously it does provide a "feel" for terminal capability. 
Evaluation of four parameter-coupled motion would require the use of a computer simulation or 
actual terminal testing and in the author’s opinion it is not worth the added effort (unless the 
commercial vendor wants to do it at his own cost). The idea here is if System A has a higher 
motion MOP score than System B, it will more than likely perform better under true operating 
conditions. 
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Figure 51. Velocity Utility Curve 




Velocity (knots) 



131 




Figure 52. Pitch, Roll, Yaw Rate Utility Curve 
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Table 21 . Mobility Sub-Attiribute Utility Values 
a) Sub-attribute Rankings 

Ranking of the data rate sub-attributes is provided in Table 22, and indicates an 
equal weighting for each measure of mobility. Equal weighting is the author’s way of accounting 
for motion coupling. In the real world each of the motions does not occur independently and 
therefore one form of motion rarely exists without another. Weighting one motion over the other 
implies that one motion has a greater impact on mission success. If this is the case and the user 
can validate the claim then unequal weighting should be used. In that case relative preference 
would be determined using AHP. 
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Ranking 


Justification 


1. Velocity, Pitch, Roll, Yaw 


All four measures are of equal importance (w =25%), and 
demonstrate the user’s desire to conduct communications on 
the move in a wide range of environments. 



Table 22. Mobility Sub-attribute Ranking 



b) Mobility Measures of Performance calculation 

Mobiity 

MOP = (w * Velocity) -t- (w * Pitch) h- (w * Roll) + (w * Yaw) 

I 1 ' 1 1 

VGlocity Pitch Roll Yaw 

Figure 53. Mobility MOP Calculation 

As indicated in Figure 53 the mobility MOP is the weighted sum of the sub- 
attributes. Given that the author’s opinion is that each of the sub-attributes are equally ranked 
Mobility MOP = 0.25*(Velocity + Pitch + Roll + Yaw). 

3. Quantity 

Determination of quantity critical values depends entirely on what the evaluation team 
deems important. The team needs to decide how the commercial system will be employed. Is it 
intended to play a limited role and augment existing wideband service, or is it intended to be the 
sole source or primary means of wideband service? The answer to this decision drives the sub- 
attribute critical values. For example, if the commercial vendor is going to be the sole supplier of 
naval wideband communications for a given theater then the total access value must exceed 
anticipated naval requirements. Furthermore, naval forces may desire to be that vendor’s primary 
or at least largest customer in that region. Being the largest customer gives naval forces more 
influence over the vendor. Under this example, a threshold total access value could be written as; 
threshold total access = [the number of required naval access -i- 25%]. This implies that all naval 
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requirements are meet and naval forces constitute 75% of the vendor’s customer base in that 
region. Clearly the Navy and Marine Corps would be able to influence the vendor. Figure 54 
represents this point of view, and reflects a drop-dead position of naval requirements accounting 
for 25% of the customer base, and an objective value of 100% of the customer base (i.e., naval 
forces are the sole customers). 




Total Accesses 



Figure 54. Total Accesses Utility Curve Given a Desire to be the Largest Customer 

On the other hand if the user’s desire is to use the commercial system as a 
complementary component to an overall wideband architecture (i.e., the commercial system is not 
the sole provider of service) then the above use of the total sub-attribute is not appropriate. 
Furthermore given the nature of the emerging commercial wideband systems it is unlikely that the 
international partnerships that support these companies would allow U.S. military forces to be 
their primary customer. In this case the total critical value only identifies a vendor’s capacity limit. 
For naval users as long as the required amount of capacity is available when it is needed then the 
vendor meets naval needs. Therefore, in this case the utility of the total sub-attribute is binary, 
either the vendor has the available capacity (utility = 1.0) or he doesn’t (utility = 0.0). This is a 
realistic scenario especially if commercial systems are being used to provide non-critical (i.e., lives 
do not dependent on it) capacity. 
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Determination of critical values for the excess capacity sub-attribute is also a function of 
how naval forces intend to use the commercial system. In the scenario where the commercial 
service is the only wideband service large amounts of excess capacity would be desired. In the 
case where the vendor-supplied service provides just one piece of the overall requirement then 
less excess capacity can be tolerated. It is clearly understood that existence of capacity does not 
mean that excess capacity will be given to naval forces. Instead in time of crisis this capacity will 
hotly competed for. The excess capacity sub-attribute only indicates how much capacity exists. 
DoD use of that capacity is certainly not guaranteed. Figure 55 provides arbitrary selected values 
for excess capacity given that naval forces are relying solely on one commercial vendor for 
service. The curve is therefore the companion curve of Figure 54. The values on the x-axis are 
provided in terms of percent above naval theater peacetime requirements. Therefore the drop- 
dead value of 10% suggests that at a minimum naval forces would like the ability to compete for 
an excess capacity that would met needs equal to 10% of existing peacetime requirements. 
Stated another way, a system that maintains excess capacity that will not meet at least 10% of 
naval peacetime requirements has 0.0 utility. Threshold and objective values indicate significant 
surge capability and imply that the commercial vendor is carrying significant amounts of capacity 
in reserve. As previously mentioned this type of action is highly unlikely. However, if the naval 
forces are intent on using a single source provider it is in the warfighter’s best interest to insure 
that surge capacity is available. In the time of war or increased tension it is not at all unreasonable 
to assume that wideband requirements would increase by 50 to 100%, if not more. [Boyd, 1997] 
Clearly the values are open for debate and future evaluators are encouraged to thoroughly 
investigate and validate critical utility values. 
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Excess Capacity (as % of naval peace tiem requirements) 



Figure 55. Excess Capacity Utility Curve 

a) Sub-attribute Rankings and Measures of Performance Calculation. 
Ranking of the data rate sub-attributes is provided in Table 23. As with the other 
sub-attributes relative preference would be determined using AHP. Figure 56 presents the formula 
for calculating the quantity MOP. 



Ranking Justification 

1 . Total The total quantity of accesses indicates as system’s theater capacity and 

therefore a maximum operational capability. 

2. Excess Excess capacity is not owned by naval forces and therefore must be competed 

for. Excess does not represent a known level of operational capability. 



Table 23. Quantity Sub-attribute Ranking 
Quantity 

MOP = (w * Total) ■+■ (w * Excess) 



Total Excess 

Figure 56. Quantity MOP Calculation 
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4. 



Capacity Measures of Performance Ranking 



Ranking Justification 

1. Quantity Indicates the user’s preference for creation of a network of users (large number 

of accesses). Increased participation (i.e., smaller amounts of input from more 
users) has greater combat effectiveness than large amounts of input from few 
users, (i.e., network centric warfare expresses this notion). 

2. Mobility The fundamental warfighter tendency to favor speed and agility of movement 

over increased information availability. 

3. Data Rate Important but not the most important of the three. Supplied capacity has 

historically lagged and will likely continue to lag user needs. Capacity for 
capacity sake is of less value than increased network participation and 
mobility. 



Table 24. Capacity MOP Ranking 

Table 24 provides a strawman ranking and justification of that ranking for the capacity 
MOPs. The fundamental element that drives this ranking is that achieving a high data rate is less 
important than increasing the number of informed users, and increased data rate should not 
negatively impact a platform’s ability to maneuver. 

F. PROTECTION MEASURES OF PERFORMANCE 

As presented in Chapter III all three of the protection MOPs are based on subjective user 
analysis. Therefore characterization of utility curves is not possible. It is enough to indicate that in 
all cases users will define characteristics that establish boundaries for acceptable (i.e., threshold) 
levels of confidentiality, authenticity, and integrity protection. Furthermore users should recall that 
there is no requirement to indicate linear utility for any of the performance measures. Figure 58 
and Table 25 are provided as points of departure for future discussions and refinement of the 
protection MOPs. 
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1.0 



utility 0-5 



0.0 





Drop Dead Threshold Objective 

Increasing Information Protection 
{Confidentiality, Authenticity, Integrity) 

Figure 57. Example Protection MOP Utility Curve 


Ranking 


Justification 


1. Confidentiality 


The most important MOP because it encompasses data security, traffic 
security, and location security. 


2. Authenticity 


More important than integrity because integrity exploitation is 
accomplished through authenticity exploitation. 


3. Integrity 


An important measure but usually requires on lack of confidentiality and 
authenticity to be effective. 




Table 25. Protection MOP Ranking 

G. INTEROPERABILITY MEASURES OF PERFORMANCE 

The discussion of interoperability in Chapter III resulted in the determination that the 
interoperability MOE directly represents system design characteristics and therefore does not 
require any MOPs. Although many elements go into defining interoperability (i.e., hardware and 
software standards) the level of detail that those elements represent is beyond that required for 
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sufficient for comparison of commercial wideband systems. The principle behind this argument is 
that the warfighter needs an interoperable system. Degrees of interoperability represented by 
fulfillment of various hardware and software standards do not imply a completely interoperable 
system. The JTA provides guidelines that if followed results in an interoperable system, or at least 
a system that conforms to accepted DoD interoperability standards. Therefore as represented in 
the figure and table (Figure 58, Table 26) adherence to JTA standards is assigned a drop-dead 
utility value of 0.0. This assignment represents the author’s opinion that in the current and 
foreseeable future naval forces will only procure IT systems that adhere to DoD interoperability 
standards. Anything less than full compliance will not suffice give the joint and fiscally constrained 
defense environment. It could be argued that this drop-dead assignment is too rigid and that 
perhaps drop-dead utility should be set at partial JTA compliance with an approved waiver. Of 
course this is an option, but it undermines DoD’s goal of seamless integrated information 
exchange. Regardless, the true utility curve will be derived from what the users and evaluators 
decide is most important and perhaps interoperability will be traded for increased performance 
elsewhere in the system. 

The threshold and objective values were assigned as a direct result of the author’s 
position that a system is expected to meet JTA requirements (i.e., drop-dead = JTA compliant). 
Basically a system’s interoperability score only increases with interoperability capabilities above 
JTA compliance. That is, threshold interoperability is assigned as the capability to interoperate 
with multiple wideband SATCOM systems in addition to being JTA compliant. Obviously the word 
multiple does not define a specific amount of vendors, and therefore it is recommended that users 
give credit to vendors according to the number of systems they interoperate with. For example a 
system that is two vendor compatible may receive a score of 0.5 while a four vendor compatible 
system may receive a score of 0.6. Despite this range of values the idea is that threshold 
requirements should truly reflect the naval users desire to possess multi-mode, JTA compliant IT 
systems. The objective value, universal terminal, is listed and likely represents an unachievable 
goal. As the name implies the user is provided a terminal that can interoperate with all known 
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wideband systems. Given that achievement of a universal terminal is doubtful (besides being cost 
prohibitive) the author expects that user utility will increase at a decreasing rate (i.e., utility curve is 
concave down, y = after threshold. 




titeroperabilHy 



Figure 58. Interoperability Utility Curve 





Interoperability 


Utility 

Score 


Drop-dead 


JTA compliant 


0.0 


Threshold 


Multi-vendor 






Capable 


0.5 


Objective 


Universal 






Terminal 


1.0 



Table 26. Interoperability MOE Utility Values 

In summary interoperability is not defined by any MOPs and therefore is represented in 
the hierarchy as a stand alone MOE. The interoperability MOE awards value to service providers 
that seek to increase interoperability across commercial boundaries. Satisfaction of naval 
interoperability requirements is accomplished if competing systems comply with JTA standards 
and it is assumed that systems will receive and interoperability score of zero otherwise. 
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H. 



ACCESS AND CONTROL MEASURES OF PERFORMANCE 



Relationship and legal MOP utility curves are presented in Figure 59 and Table 27. Both 
relationship and legal are measured in terms of user determined risk and for ease of presentation 
are displayed on the same curve. Although both curves display discrete linear utility it is important 
to recall that the MOPs are continuous and it is assumed that evaluators will modify the curves to 
reflect actual user desires. 




Relationship & Legal Risk 



Figure 59. Relationship and Legal MOP Utility 





Relationship 


Legal 


Utility Score 


Drop-dead 


High Risk 


High Risk 


0.0 


Threshold 


Medium Risk 


Medium Risk 


0.5 


Objective 


Low Risk 


Low Risk 


1.0 



Table 27 . Access and Control MOP Utility Values 

1. Access and Control Measures of Performance Ranking 

It is the author’s opinion that relationship risk out weighs legal risk. Or stated another way 
legal risk can be influenced by a consortium’s political relationship. If a consortium is considered 
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friendly then within the legal gray areas the tendency maybe to give the benefit of the doubt to the 
U.S. In essence if the consortium doesn’t deem a user’s actions as illegal then they aren’t illegal 
and service will not be denied. Although this statement skirts along the borders of ethics it reflects 
political reality; allies assist each other. This works both ways, depending on the relationship with 
the vendor. The medium risk organization may not be willing to give U.S. forces any “slack” and 
may choose to interpret legal restrictions in a rigid manner. 

Ranking Justification 

1. Relationship Political relationship drives how legal restrictions are interpreted. 

2. Legal Enforcement of legal restrictions are significantly influenced by politics. 



Table 28. Access and Control MOP Ranking 

I. QUALITY OF SERVICE MEASURES OF PERFORMANCE 

The discussion of QoS Chapter III resulted in the presentation of six MOPs; robustness, 
availability, delay, blockage, data integrity and weather. This section discusses and proposes the 
critical utility values and final ranking of the MOPs. The reader is reminded that because a user 
survey was not conducted linear utility is assumed. 

1. Robustness 

As presented in Chapter III robustness reflects the existence of an automated customer 
priority scheme. It is the author’s opinion that this MOP is binary, either the vendor has or does not 
have an automated priority system. Therefore existence of an automated scheme receives a fully 
utility value of 1 .0, and non-existence receives a score of 0.0. It could be argued that a threshold 
value of 0.5 could be added to reflect existence of a manually executed priority scheme. However, 
given the naval user’s need for consistent service manual priority routing is likely to not be 
responsive enough to meet naval needs. Of course representation of robustness utility is 
ultimately up to the user and therefore should be debated prior to system evaluation. 
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2 . 



Availability, Delay and Blockage 



Availability is measured in terms of probability of access, P(access), while delay and 
blockage are measured in terms of P(no delay) and P(no blockage). A value for each of these 
MOPs will most likely be obtained using some form of system simulation. Evaluators must either 
identify a simulation for vendor’s to use or subjectively critique the vendor created simulation to 
identify any potential biases. The remaining alternative is for the evaluators to create and run their 
own simulations. Regardless of the method used evaluators must know and understand the 
model’s strengths, weakness as well as the assumptions made during its creation. The critical 
values presented in the utility curves below (Figure 60, Figure 61, Figure 62 and Table 29) are the 
derived from the author’s interpretation of naval needs and are certainly subject to debate and 
refinement by a future evaluation team. 




Avalabiiity (% of time) 




Figure 60. Availability Utility 
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7-1 




No Delay (% of time) 

Figure 61. No Delay Utility 




No Blockage (% of time) 

Figure 62. No Blockage Utility 





Availability 


No Delay 


No Blockage 


Utility Score 


Drop-dead 


95% 


90% 


90% 


0.0 


Threshold 


97% 


95% 


97% 


0.5 


Objective 


99.9% 


98% 


99.9% 


1.0 



Table 29. Quality of Service MOP Utility Values 
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3. 



Data Integrity 



1 

0.9 
08 
0 7 
06 
0.5 
0.4 
03 
0.2 
0.1 
0 



Figure 63. Data Integrity Utility 

Data integrity is measured in terms of BER. BER determines what products and 
applications the customer can use. Acceptable BER should be entirely user determined and 
needs driven. Table 4, page 38, lists the acceptable BER for different applications. Critical values 
for the data integrity and justification for those values are provided in Figure 63 and Table 30. 






Data Integrity 
(BER) 


Utility Score 


Justification 


Drop-dead 


IxlO'" 


0.0 


Requirement for efficient operation of 
computer networks. 


Threshold 


1x10’’° 


0.5 


The current GBS requirement. 


Objective 


Ixio’^ 


1.0 


An objective value that reflects the user’s 
desire for uncorrupted data. 



Table 30. Data Integrity Utility and Justification 
4. Weather 

As presented in Chapter III, the weather MOP is reflected as P(available path) for a given 
rain intensity. In order to accurately compare vendor weather claims the evaluator needs to 
provide the vendor with a rain climate region. The most frequently used rain attenuation model is 
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the Crane Model, which “provides median distribution estimates for eight rainfall regions, A-H, 
covering the entire globe.” [Freeman, 1991, pp. 512] Evaluators should provide the vendor with a 
standard terminal located in a selected rain climate region. As a point of reference region A 
covers to the poles and indicates a dry tundra region while region H refers to wet tropical regions 
of the earth (i.e., Amazon rainforest). It is anticipated that since most of the wideband systems 
intend to operate in the same frequency band P(available path) will be very similar. Differences 
will result from available link margin and path diversity due to differences in the numbers of 
satellites for each system. As with the other MOPs presented in this section weather critical 
values should be debated and refined by future evaluators. Figure 64 and Table 31 show the 
weather critical values. 




Figure 64. Weather Utility 





Weather 

P(available path) 


Utility Score 


Drop-dead 


97.0% 


0.0 


Threshold 


98.0% 


0.5 


Objective 


99.9% 


1.0 



Table 31. Weather Utility 
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5. Quality of Service Measure of Performance Ranking 

Recall that the AHP weighting method is not used within this thesis. However the author 
provides a strawman ranking (refer to Table 32). Also it is important to remember that the ranking 
does not provide a value ratio. The fact that robustness is ranked as the number one concern 
does not provide any indication of its weight relative to the other MOPs. In conclusion, the 
strawman ranking is intended to force users to discuss the relevant issues and flush out the 
significant details. 



Ranking 


Justification 


1. Robustness 


Is the highest rank MOP because without an automated priority service 
scheme availability, blockage and delay as defined in this hierarchy have 
little value. 


2. Availability 


The second ranked MOP because the remaining MOPs have no value if 
the user does not have access to the system. 


3. Blockage 


More important that the remaining values because a message that does 
not get to the user is worthless. 


4. Data Integrity 


If a user has priority, access and is not blocked then the message should 
be transmitted uncorrupted. 


5. Delay 


In terms of likelihood of occurrence a message delay is more likely than 
reduction in service due to rain fade. 


6. Weather 


The least important QoS MOP because it has the smallest impact over the 
life of the system. 



Table 32. Quality of Service MOP Ranking 



J. FLEXIBILITY MEASURES OF PERFORMANCE 

Recall that flexibility is intended to represent a system’s ability to support the full dynamic 
range of military operations, missions, and environments. The "ability to support" can be 
characterized by the eleven traditional logistics elements plus an additional factor called 
upgradability. However for use within this hierarchy only three elements were presented as MOPs 
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and used to represent the flexibility MOE; operational availability, reliability, and maintainability. 
This section discusses and proposes the critical utility values and final ranking of the MOPs. It is 
recommended that the user evaluate the other logistics elements for potential use as MOPs. 

1. Operational Availability 

As presented in Chapter III operational availability Ao reflects how mission capable a 
system is. It is the author’s opinion that this MOP is binary, either the system is full mission 
capable (FMC) or it is not. Of course representation of Aq utility is ultimately up to the user and 
therefore should be debated prior to system evaluation. It is important that prior to evaluating 
systems, users determine the Ao scores that represent the FMC, PMC and NMC ranges. Although 
scores below FMC will not have value (utility = 0.0) within the hierarchy defining PMC and NMC, it 
is important so evaluators can provide feedback to vendor's. User's will have to carefully select a 
simulation, or clearly articulate the parameters of a simulation that can be used to generate Aq 
scores. If a simulation is not used and the hierarchy is employed as a "first cut" evaluation then 
vendor provided Ao values accompanied with calculation assumptions can be used. In conclusion 
as a binary MOP, an Aq score of FMC would receive a utility score of 1 .0, and a score of not FMC 
would receive a score of 0.0. 

2. Reliability 

Reliability represents the number of hours without a critical failure under specified mission 
conditions within the hierarchy. It is reflected as mean time between operational mission failure 
(MTBOMF). The goal is to specify how often the user terminal will not be mission capable due to 
failure of some component. Again for accurate head to head comparison of different systems 
evaluators may have to mandate use of a particular simulation, however first cut comparison can 
be done using vendor provided values. When vendor values are used recall that those values are 
derived for the ’’home user’s environment” and not the deployed naval environment. The utility 
values presented in Figure 65 and Table 33 represents the author's recommendations and are 
loosely derived from GBS system requirements. Future evaluators are not only encouraged to 
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investigate these values and refine them as required but also to accurately reflect their non-linear 
utility. 




Figure 65. Reliability Utility Curve 





Reliability 
(MTBOMF hours) 


Utility Score 


Drop-dead 


2500 


0.0 


Threshold 


3000 


0.5 


Objective 


5000 


1.0 



Table 33. Reliability MOP Utility Values 



3. Maintainability 

Recall that in Chapter III, maintainability is defined as "the ability of an item to be retained 
in or restored to specified condition when maintenance is performed by personnel having 
specified skill levels. Using prescribed procedures and resources, at each prescribed level of 
maintenance and repair." [Hoivik, 1998] Further it was proposed that since evaluators will most 
likely be comparing "paper concepts" that maintainability be measured as a spectrum (refer to 
Figure 40, page 102). Users must debate and determine the characteristics that define poor, 
acceptable and outstanding maintainability. This measure is clearly subjective and reflects the 
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user’s opinion of how maintainable a system based on intended maintenance concepts. Table 34 
relates the maintainability spectrum to critical utility values. 





Maintainability 


Utility Score 


Drop-dead 


Poor 


0.0 


Threshold 


Acceptable 


0.5 


Objective 


Outstanding 


1.0 



Table 34. Maintainability Utility 



4. Flexibility Measures of Performance Ranking 

Proposed ranking of the three flexibility MOPs is provided in Table 35. It is assumed that 
actual use of the hierarchy will include the use of AHP to properly perform ranking and weighting. 



Ranking Justification 

1. Operational Availability The most critical parameter of the three because it reflects 

when the user can employ the system. 

2. Reliability More important than maintainability because reliability drives 

corrective maintenance. 

3. Maintainability Critical because it directly impacts the naval forces ability to be 

self supporting. 



Table 35. Flexibility MOP Ranking 

K. MEASURES OF EFFECTIVENESS RANKING 

As originally stated the author’s intent is to provide a strawman hierarchy. Within this 
strawman AHP was not used to generate relative rankings (weighting) instead the author provided 
ordinal rankings for each MOE. This has also been done for the MOEs and is displayed in Table 
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36. Recall that this ranking does not provide any indication of relative value and is intended as a 
point of departure for AHP discussions. 



Ranking 


Justification 


1. Access and Control 


Assured access is a fundamental and the most important naval 
user requirement, (coverage and capacity without access is 
useless). Control over user access to include the ability to deny 
access to other users is a critical naval requirement. 


2. Coverage 


Without coverage (as defined in this hierarchy) the user does 
not have any capacity. Furthermore a system’s coverage 
dictates the geographic regions in which a naval user can use 
the system. 


3. Capacity 


The fundamental reason that naval users desire wideband 
systems is because of their high capacity. Without this high 
capacity there is no reason to even investigate the use of 
commercial wideband systems. 


4. Protection 


Protection of naval user confidentiality, authenticity and integrity 
are basic user requirements that are critical to mission success. 


5. Quality of Service 


Naval users must have the ability to transfer information in a 
timely and accurate manner. 


6. Flexibility 


The logistics requirements of a system must not impede the 
operational utility of the system. 


7. Interoperability 


Any selected system must meet joint interoperability standards. 




Table 36. Strawman MOE Ranking 
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V. 



IMPLEMENTATION OF THE RESULTS 



A. RECALLING THE PROCESS 



1) 

2) 

3) 

4) 

5) 



6 ) 

7) 

8 ) 

9) 

10 ) 



The Process 



Define top level question. 

Accept CRD required characteristics as 
functional objectives and therefore as MOEs. 
Create a user profile. 

Using the user profile identify and define MOPs 
for each functional. 

Create strawman utility curves for each MOP. 
a) Utility 



b) 



Dropdead =0.0 
Threshold = 0.5 
Objective - 1 .0 
Performance 

i) Dropdead = no value 

ii) Threshold = minimum acceptable 

iii) Objective = desired 

Use AHP to pair-wise compare MOPs and 
generate weights. 

Use AHP to pair-wise compare MOEs and 
generate weights. 

Perform sensitivity analysis to examine the 
effects of different weighting schemes and utility 
functions. 

Introduce CAIV by conducting a market survey 
to create a SEV vs. Cost curve. 

Evaluate a system. 



Solution Effectiveness Value 

SEV = S(wi*MOEi) 



Functional Objective 

MOE = E(wj*MOPj) 



Measures of Performance 
MOP 



Figure 66. The Evaluation Process 

This document does not attempt to provide an all-inclusive list that applies to every naval 
wideband issue. Instead the "strawman" hierarchy should serve as a point of departure for future 
naval wideband discussions. The primary intent of the evaluation hierarchy presented in this 
research is; 

• That it be used as a starting point by DoD (Naval) integrated product teams (IPTs) that 
are investigating COMMER wideband systems. 
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• That it be used by COMMER SATCOM providers as a way to identify how their system 

meets or fails to meet naval needs. 

B. USE BY A DOD INTEGRATED PRODUCT TEAM 

First and foremost the IPT must define the top level question. Given the question the 
focus should then be directed towards creation of a user profile. The level of detailed provided by 
the profile is entirely a function of how far the IPT wants to investigate and characterize the user. 
Defining a profile and subsequent translation of user characteristics to SATCOM system design 
parameters (MOPs) will be most effective if the IPT contains a balanced mix of operators and 
SATCOM specialists. The operators bring the user needs and the specialists provide the technical 
knowledge. Together they should be able to cooperatively develop MOPs that reflect user needs 
and have meaning to both designers and users. It is important to point out that the MOPs and 
MOEs presented should be debated and challenged. They may not be applicable to every 
independent situation. However, use of the hierarchy as a starting point and framework for the 
analysis will provide structure and a common focus. A market survey should be conducted in 
conjunction with or prior to refinement of the source selection hierarchy. Utility curves need to 
reflect user preference and the current state of technology. If the user truly intends to seek cutting 
edge performance then utility should reflect that desire. Additionally, utility should not over reward 
old or passing technologies. Once a refined hierarchy is developed sensitivity analysis should be 
used to determine if and how the weighting schemes bias the SEV. If bias is found and is not 
intended then the team should re-structure the hierarchy. As a final effort (refer to Appendix A) the 
refined and completed evaluation hierarchy should be freely given to the competing commercial 
vendors. Source selection should be accomplished through comparison of competing alternative 
SEVs. Furthermore, the team should compare their independently developed SEV for a particular 
system with the SEV provided by the particular vendor. This will give the team some visibility of 
vendor injected bias. Source selection should not be solely based on the team’s SEV results, 
however SEV should have considerable weight in the decision process. 
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c. 



USE BY A COMMER SATCOM SERVICE PROVIDER 



Although the commercial (civilian) SATCOM community would have a difficult time 
creating an evaluation hierarchy with utility and weighting schemes that exactly reflect naval 
warfighter needs, some usefulness still can be derived from the parameters presented in this 
document. At the very least they provide the commercial vendor a more focused and defined view 
of naval requirements and how they relate to SATCOM design parameters. Although the topics 
presented in this research are not "new," at a minimum the document serves as a single point of 
reference for naval wideband SATCOM concerns. Furthermore, an enterprising vendor can use 
the hierarchy to structure a solicitation response. In blunt terms this document connects SATCOM 
performance parameters to concepts that mean something to the naval warfighter. This enhances 
the vendor’s ability to present his system in a manner that appeals to the user’s definition of value. 
The bottom line, the hierarchy can be used to assist in the marketing and sale of service. 

Additionally, as presented in Appendix A, access to the user’s evaluation hierarchy gives 
the commercial designer insight into the naval user’s needs. This insight allows the designer to 
make value added trades that increase the user’s perception of system effectiveness. Potentially 
these design trades may come at no or little cost to the designer. That is, the designer may have 
the ability to include naval needs with out compromising original system (commercial) priorities. 
Furthermore, as history has repeatedly demonstrated, the military need for advanced technology 
(i.e., aircraft, Internet, GPS, the SATCOM industry itself), often leads commercial interest and 
need for the same technology. Therefore, the evaluation hierarchy could also be used as a 
commercial market-forecasting tool. Most of the performance measures presented represent 
commercial as well as military needs (i.e., international finance requires levels of protection equal 
to or greater than military requirements). In the cases where military needs exceed current 
commercial requirements the author’s opinion is that it only reflects an immature under-developed 
market. For example, the Navy’s use of tele-medicine for aircraft carriers at sea^® demonstrates 

Currently being conducted using the commercially provided Challenge Athena program. 
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the ability to conduct wideband communications with a mobile platform. Is there a commercial 
market for this technology? Yes, potentially a wideband terminal could be installed in domestic 
ambulances and once on scene the medical personnel could do an "at the crash site" video 
consult with an emergency room doctor located at the receiving hospital. If phase array antenna 
technologies progress and satellite regulatory restrictions^^ are modified the capability could exist 
to conduct a video consult while the ambulance is enroute to the hospital. One last example is the 
potential desire for company executives to be able to conduct two way video tele-conferences 
while in-flight to an overseas or cross-country destination. Both of these ideas represent needs 
that the military currently has and desires to fulfill (i.e., highly mobile real-time video for 
collaborative planning and situational awareness). Therefore use of the naval evaluation hierarchy 
can assist in forecasting emerging commercial uses and therefore should be used to influence 
spacecraft and constellation design. 



2'' In most cases wideband fixed-satellite service (FSS) has priority over wideband mobile-satellite 
service (MSS) in terms of frequency (spectrum) conflicts. Therefore, mobile wideband service 
although it may become technologically possible is restricted to operating on a "not to interfere" 
basis with FSS. Unless modified this international regulation limits the commercial potential of 
mobile wideband communications. 
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VI. 



CONCLUSIONS 



A. THESIS SUMMARY 

The evaluation hierarchy presented in this thesis is not intended to be a one size fits all 
solution. It was recognized that the selection of commercial SATCOM service is a multi-variable 
decision and that attempts to account for every possible combination of design parameters often 
result in a sluggish and ineffective evaluation. With this spirit in mind it is intended that the 
performance parameters presented in this thesis will be used to focus future groups intent on 
evaluating commercial wideband providers. 

In this thesis the author presented commercial wideband performance evaluation 
measures that were: 

• tied to user needs, 

• easy for decision makers to understand, 

• quantifiable, 

• and sensitive to system design thereby highlighting differences in competing concepts. 

[SMAD, 1992, pp. 60] 

Critical to the usefulness of these performance measures is the identification of a user 
profile and the construction of an evaluation hierarchy. A naval user profile was constructed using 
joint and naval doctrine and vision publications. A proposed evaluation hierarchy that articulates 
users needs through user defined utility and weighting was presented in Chapters III and IV. The 
hierarchy allows evaluators to incrementally sum the weighted performance measures up each 
level of the hierarchy ultimately leading to a system effectiveness value. This value can be used to 
compare the relative abilities of competing systems to satisfy user needs. Ideally, this proposed 
evaluation hierarchy will be used by future naval SATCOM integrated product teams to evaluate 
competing commercial wideband systems intended for naval use. 
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B. RECOMMENDATIONS FOR FUTURE RESEARCH 

Across DoD a significant amount of resources have been dedicated to the search for 
solutions to meeting the Navy and Marine Corps future wideband SATCOM needs. Regardless of 
the method, selection of a system ultimately comes down to comparison of alternatives based on 
their perceived value. Perceived value is appropriate because system value should be a 
subjective measurement defined by the warfighter and directly tied to his needs. Therefore any 
efforts that maintain the same central focus of evaluating commercial technologies using 
warfighter derived needs, whether they expand on or criticize the results presented in this thesis 
has value to the naval forces. Specific areas of related research include: 

• Conduct a market survey to define the relevant SEV utility curve. 

• Perform a source selection or system comparison using the hierarchy presented above. 
This includes refinement of user utility curves and MOP and MOE weighting. For 
example, answer the question. What commercial system best meets USCINCPAC’s 
wideband needs? 

• Use the hierarchy to evaluate commercial systems that could DoD could directly purchase 
and subsequently own and operate, (i.e. meet the Wideband Gapfiller need). 

• Evaluate current and emerging commercial wideband systems using the hierarchy, (i.e., 
Astrolink, Hughes Spaceway, Loral Cyberstar, Skybridge, Teledesic, and West). 

• Modify the hierarchy to reflect user needs and the trade space required for the design of a 
DoD owned and operated wideband SATCOM system, (i.e.. Wideband Gapfiller). 

• Work with a commercial vendor to see where system modifications can be made to better 
meet naval needs as reflected in the hierarchy. 

• Refute the hierarchy above and provide a better solution for representing naval wideband 
SATCOM needs in system selection. 
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APPENDIX A. COST AND COST AS AN INDEPENDENT VARIABLE 



The entire focus of this thesis has been on system performance. The reality of today’s 
fiscally constrained environment is that DoD can not afford to pay for increased performance for 
performance sake. The notion of cost as an independent variable (CAIV) is used to inject cost 
considerations further into the acquisition decision process. There are numerous definitions and 
methodologies for using CAIV in the acquisition process. This appendix presents the author’s 
approach (opinion) to the use of CAIV in the acquisition of COMMER wideband SATCOM service. 

As with any program, balance between cost, schedule and performance must exist. 
However, it is also true that balance does not imply that all three are equally weighted. Most would 
argue that whatever the weight, performance is the most important (highest weighted) of the 
three. The thrust of this argument is that for military systems performance is critical; lives are at 
stake, poor performance kills. Therefore the focus on performance begins with requirements 
definition and extends through system evaluation. It is the author’s opinion that a program must 
maintain this ultimate singular focus. This does not imply that cost and schedule are not 
important. What it does insinuate is that the focus should be to maximize performance 
constrained by cost and schedule. This statement is different from the two-focus strategy of 
maximizing performance while minimizing cost. T rying to execute the two-focus strategy implies 
that performance and cost are both simultaneously goals and constraints. While maximizing 
performance, cost is constrained, which confuses attempts to minimize cost while constraining 
performance. The contradiction resides in the fact that performance can not be maximized 
(optimized) if it is simultaneously a constraint. Furthermore, cost is not an /ndependenf variable if 
it is both a constraint and a variable trying to be optimized. When performance is used as a 
constraint a portion of the entire range of possible values is "out of bounds", "off limits" and 
therefore can not even be considered. When used as the goal, (i.e., maximize performance) the 
entire range of values are considered. Once considered values are ruled out if they do not fall 
within the constraint requirements. Although slight, the difference between the single and dual 
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focus strategies is significant. The analogy of an employee having two bosses with equal authority 
exemplifies this difference. The first boss is performance oriented, the second is cost motivated, 
the employee is stuck in the middle often torn in conflicting directions. The idea of a trying to 
simultaneously maximize performance while minimizing cost for a particular system creates the 
same confusion. Simply stated, a single unified focus (i.e., unified commander) reduces confusion 
and facilitates the efficient execution of system trades. For military use systems, where lives 
depend on effectiveness, performance must be the driving focus. Cost and schedule should be 
seen as constraints and not as optimization goals. 

The focus on performance extends to the handling of sub-standard performance. It is the 
author’s opinion that below threshold performance requirements should not be accepted as a way 
to reduce costs or schedule.^^ Not only can below threshold performance threaten lives but below 
threshold systems can actually increase dollar costs. Opportunity costs increase since time and 
money are wasted on a system that doesn’t meet user needs. The resources would have had 
more value if committed towards another program. Dissatisfied users will at the least under utilize 
and at the worst reject the sub-standard system. Any savings that were forecasted to occur due to 
the fielding of the "improved" system will not be realized. Often times these costs are not easily 
identifiable because they are hidden in increased man power requirements related to logistics 
support and maintenance of the new system. The bottom line is that performance compromises 
that extend below user designated thresholds should not be done. If the applicable constraints 
(i.e., cost) prohibit meeting or exceeding threshold performance requirements the program should 
be canceled or delayed until the constraints are adjusted. 

Given a single performance-oriented focus and unwillingness to compromise for below 
threshold performance the next step in the process is to clearly specify the user’s performance 
utility. The user should know and understand his performance utility and therefore be able to 

This statement is made knowing that in all cases trades are a reality and that in some 
instances below threshold performance in one area must be accepted in order to achieve 
threshold or objective performance in another. However the desire to maintain across the board 
threshold or above performance should be the driving focus. 



160 



associate service (system) value with cost. That is the user should be able to state that an 
incremental increase in performance of X% is worth Y dollars and Z time. This is how the cost 
performance trade is conducted, with cost introduced as an independent constraint (independent 
variable) not as the goal of the problem. Again this does not imply that fiscal limits can not be 
imposed. In fact the word constraint clearly implies that costs are and can be restrained. The user 
creates their utility curve after conducting a detailed market survey. The market survey should be 
used to estimate a realistic threshold cost position. Threshold price is a function of market 
conditions and the state of technology and is critical to the creation of an accurate cost utility 
curve. Using the threshold cost as a starting point the next step is to determine how much 
more/less the user is willing to pay for an incremental increase/decrease in performance. 
Characterization of the utility curve should extend from the drop-dead performance requirements 
through the threshold requirements and up to the objective requirements. 




Figure 67. Suggested User Utility Curve 
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Figure 67. Suggested User Utility Curve, represents what the author projects a typical utility 
curve for the acquisition of a commercial service or system may look like. In general the curve 
should have the following characteristics: 

• The user’s utility cun/e is non-linear. (Figure 67, provides a linear curve for reference). 

• Performance (SEV) below threshold requirements has very little utility, but as SEV 
increases utility increases at an increasing rate. (Represented in Figure 67 as Utility = 
(SEV) ^). 

• As performance (SEV) approaches the objective level utility increases at a decreasing 
rate. (Represented in Figure 67 as Utility = (SEV) ’^^). 

• Utility of an objective system = 100% (SEV =1 .0) 

• Utility of a threshold system = 50% (SEV =0.5) 

• Utility of a drop-dead system = 0% (SEV = 0.0) 

A primary assumption in the acquisition of a commercial service or system is that the DoD 
may not be the primary user of the commercial item. As a minority user the DoD’s ability to 
influence the design may be limited. This does not imply that the DoD should accept sub- 
threshold service, neither does it imply that DoD won’t be a powerful customer. However, given a 
reduced ability to directly influence the design of a system DoD should seek to introduce 
incentives that encourage vendor upgrades that reach towards objective requirements and further 
satisfy DoD needs. In an effort to direct the vendor towards system improvements in which DoD 
places high value, the author believes that DoD should reveal its utility curves to the vendor. 

The idea of showing the vendor how the user assigns system value (utility) is traditionally 
seen as reducing the purchaser’s price negotiating power, and therefore a naive thing to do. The 
author agrees that in telling the vendor how the user assigns value the user does give away a 
portion of the negotiating position. However, given that for a commercial system the DoD user is a 
minority user the DoD’s negotiating power is already significantly diminished and therefore DoD 
must seek ways to encourage vendor innovation. 

By revealing user utility DoD will introduce incentives. The informed vendor understands 
that, an upgrade that meets X additional requirements is worth Y to the user. Now it is up to the 
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vendor to identify the areas in which to make improvements. If user utility (value) curves are 
accurate the vendor will attempt to incorporate system enhancements in which he receives the 
most return (i.e., those areas with the highest utility for smallest investment). Let the vendor travel 
down the user’s system evaluation hierarchy and identify functional objectives and measures of 
performance that give him the "largest bang for the buck." The vendor knows his own product 
development process and should know where he can find or generate efficiencies in that process. 
Let the vendor match user utility with his own utility and leverage the two, to produce a mutually 
beneficial product or product upgrade. 

As previously indicated the naval user’s wideband SATCOM utility curve should be non- 
linear, and its non-linearity should be a function of market factors and the state-of technology. 




Figure 68. Calculation of Increased Performance Costs 

The closer threshold requirements are to state-of-the-art the steeper the slope will be for 
performance greater than threshold. Vendor’s will have to be enticed with monetary incentives to 
surpass and go beyond state-of-the-art to reach towards objective requirements. This argument is 
justified given the greater risk and therefore cost associated with development of cutting edge 
technologies. This is represented in Figure 68. Calculation of Increased Performance Costs. The 
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non-linear utility curve implies that the user’s market survey indicated that threshold requirements 
are state-of-the-art and that increased demands for performance will require significant 
investment, and therefore significant monetary incentive for small increases in performance. The 
linear utility curve insinuates that threshold requirements are not state-of-the-art and that 
progression towards objective performance can be achieved with a proportional increase in 
funding. Again the exact shape of the curve is determined by a rigorous market sun/ey. 

An opposing argument to the idea of "openness" and sharing of utility (value) curves is 
that once revealed the vendor will use the information to exploit the user. For example, assume 
that the user’s utility curve indicates a willingness to pay a 25% premium for an incremental 
increase in performance. After conducting a thorough review the vendor realizes that the 
improvement can be accomplished for a 10% increase in overall expenses. The vendor is now in 
the position to potentially make a 15% gain by implementing the change. Some argue that this is 
not fair the vendor should reduce his price because he is now making too much profit. The author 
argues that since the user receives the upgrade at a price he originally considered fair (with in the 
user’s utility) the vendor should be allowed to retain the profit.^^ Additionally, the user should be 
thrilled that the company providing his product is so efficient and lean that it is able to out perform 
the market (perform better than predicted by the market survey) and provide increased service at 
less cost than competitors. The user should now return to his utility curve, adjust it to reflect the 
new market conditions (i.e., the providing company raised the "quality bar"), and resubmit them to 
the vendor, so that the vendor can identify other areas of mutually beneficial improvement. 

Another negative that is presented is that users may understate or inaccurately state utility 
in order to control (limit) vendor profit. Understatement of utility is counter-productive in that it only 
serves to reduce the potential for development and implementation of upgrades. From the 



33 The author recognizes that this argument is presented without validation or supporting 
evidence. Interested readers are referred to literature that discusses the theory of optimal 
contracts, and principal agent theory. 
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vendor’s point of view understated utility is seen as an inadequate, not worth the risk, incentive. 
Therefore, effective utility should accurately reflect market conditions. 

There are those who will argue that the process of continually encouraging improvement 
only serves to ratchet up user costs to a point beyond the user’s fiscal reach. Keepers of this 
mindset often fail to account for the reality of the commercial (high tech) market; product 
improvement reduces costs for threshold requirements. That is, threshold requirements are 
generally "less than cutting edge" requirements, and therefore provided at a lower cost than 
objective requirements. As a user pays for product improvements the basic cost for the threshold 
system should decline, therefore total costs should remain relatively flat. Furthermore, as system 
enhancements are made the overall "quality bar" among commercial vendors raises. This 
increase in quality attracts more commercial customers (recalling that the DoD user is not the sole 
customer) and increased competition among the commercial providers. More customers and 
increased competition generally results in decreased pricing. This is where the DoD will gain 
negotiating leverage. Increased competition implies an increased vendor willingness to negotiate 
with a valued customer. Having established itself as a valued customer DoD should be able to 
contract for improved threshold rates and/or services. The underlying premise is that the user’s 
utility curve stays tied to the market and is continually updated. 

In summary, it is the author’s opinion that in the acquisition of COMMER wideband 
SATCOM service CAIV implies: 

• That the goal should be to maximize performance constrained by cost and schedule. 

• If cost constraints prohibit greater than or equal to threshold performance then 1) don’t 
build the system or 2) wait until constraints change.^’^ 

• The user should conduct a thorough market survey to establish a realistic utility curve 
and threshold system cost. 

• Vendors should be given the user’s evaluation hierarchy including the user’s utility curves. 

Again it is recognized that this hard line, "either you have or you don’t" methodology is not 
always practical. In some cases slightly below threshold performance can be accepted as 
compensation for increased performance elsewhere in the system. 
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• Let the vendor demonstrate how his system best meets naval needs at an acceptable 
cost. Let him demonstrate how he can tweak, refine and enhance his design to better 
fulfill naval needs. 

• Let the vendor retain the gains that he creates by fielding a product upgrade. 

• Continually update user utility to reflect increases in product quality and decreasing 
market costs. 

• Seek negotiating advantages resulting from increased market competition. 

• Remain a valued customer through open sharing of user requirements and utility. 



Despite the fact that the case presented describes an ideal solution void of numerous 
influencing factors (i.e., politics) which would impede the effective implementation of such a 
system the fundamental theme still remains; a single performance-oriented focus coupled with a 
desire to encourage innovation through openness will produce a better product. 
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APPENDIX B. THE SYSTEM ENGINEERING AND EVALUATION REGIME (SEER) 

This paper was taken in its entirety as it was presented to the American Institute of 
Aeronautics and Astronautics (AIAA) and is not the work of the thesis author. It has been declared 
a work of the U.S. Government and is not subject to copyright protection in the United States. 
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Abstract 



A five stage methodology is presented describing systematic and quantitative analysis approaches 
for determining contributions alternative candidate space systems make in support of modern 
naval warfare tasks. This analysis process and the companion software may also be used to 
perform similar analysis on any requirements or needs-driven military or civil analysis task. The 
analysis methodology is developed following a specific application, the analysis of space system 
support for needs derived from littoral and expeditionary naval warfare. The first stage of the 
analysis bounds the problem by establishing key relationships among military requirements and 
space system capabilities. The second stage introduces the impact of future weapons, platforms 
and tactics. Stage three synthesizes the prior results into a system of measures for a future time 
frame. Stage four provides the decision maker with a system analysis tool to help define an 
acquisition strategy. The final stage assesses the systems’ initial and life-cycle costs and provides 
a final benefit-cost trade-off analysis tool. This methodology has been developed into a software 
program that operates as an application under Microsoft Excel. Refinements to both the 
methodology and software are planned. 



Introduction 



The importance of support from an appropriate mix of space systems acting across the spectrum 
of modern warfare is seldom questioned and has indeed been amply documented in our nations 
most recent conflicts. Accounts of Desert Storm describe the key role played throughout the 
conflict by satellite communications, navigation, and other space systems. 

For many years the United States Navy has viewed itself as the biggest “user” of space systems 
among all of the services. This assertion will not be questioned here and is intuitively agreeable, 
however, it would be extremely useful in determining the right mix of supporting space systems to 
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know precisely how modern naval warfare missions and space support are related. These 
relationships and the measures for analyzing the implications of these relationships should be 
developed and described quantitatively where possible. 

This paper documents both a methodology and a software tool developed to help formalize the 
analysis process. The Aerospace Corporation working with the Naval Space Command, 
developed a customer focused, benefit-cost analysis concept. The associated software tool is 
called Systems Engineering and Evaluation Regime (SEER) and was developed to operate as an 
application under Microsoft Excel. The methodology will be described here in the context of its 
application to the analysis of space system support for needs derived from littoral and 
expeditionary naval warfare. 

Unfortunately, established capital investment analysis techniques traditionally and effectively 
employed by businesses in making capital investment decisions (e.g. Equivalent Worth, Rate of 
Return and Replacement Cost Analysis) are not equally adaptable to analysis of government 
investments. This of course is the case, because of the vastly different objectives businesses and 
government organizations define for themselves. Businesses nearly always seek to either 
maximize profits or minimize costs. Government organizations, including the Department of 
Defense (DoD), usually seek maximum public benefit or utility at an acceptable cost. Benefit-cost 
analysis therefore becomes the capital investment analysis technique most adaptable to DoD 
investment decisions. With shrinking DoD budgets, proper investment strategies become critical 
to providing the best possible warfighter support for each available dollar. 

Five stages are used to develop a benefit-cost ratio. The first four stages focus on the benefit part 
of the ratio while the fifth stage includes cost considerations and completes the benefit-cost 
analysis. 
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The concept of “customer"’ focus is central to the successful application of this methodology. By 
this we mean that early, meaningful, and continued dialog with the Navy grass roots end users 
supported by the analyzed space systems is essential. This customer focus is also central to the 
SEER tool’s flexibility. Through user initiated changes in the analysis parameters and judicious 
selection of the missions and systems examined other services, the DoD and other agencies can 
utilize the process and the SEER tool. 

Figure 1 illustrates schematically the process we have practiced and advocate as a means of 
maintaining the desired “customer"’ focus. 

Framing the Analysis 



Analysis of space systems support to naval warfare must be firmly anchored in and rigorously 
traceable to the post cold war doctrinal changes described in “...From the Sea” and “Forward... 
From the Sea.” This axiom is often overlooked in analysis of space systems support to warfare 
missions. This premise is a fundamental element of this analysis and the methodology developed 
to conduct the analysis. 

With the collapse of the Soviet Union and the end of the Cold War, the U.S. military began a 
process of redefining its global mission and warfighting strategies. For the Navy, this meant a 
transition away from a long standing blue water, War-at- Sea doctrine. Instead of planning for 
large scale conflict with a powerful Soviet navy on the world’s oceans, the Navy’s focus had to 
shift to operations against lesser (but still dangerous) adversaries in regional contingencies of 
various magnitudes. Missions such as embargo and sanction enforcement. Non-combatant 
Evacuation Operations (NEO) , and shallow water Anti-Submarine Warfare (ASW) have replaced 
choke point control and outer air battle fleet defense. 

These fundamental changes raise the question, what sort of support is required from space 
systems in this new warfighting environment? Littoral warfare is characterized by the projection of 
seapower against an enemy’s military and strategic assets, launched from ships and naval 
platforms operating in shallow, restricted waters near to shore, and including all associated and 
supporting operations. Intuitively, it would seem that the requirements for space system’s support 
to this kind of warfare would be different from that required for blue-water combat against an 
opposing superpower. Certainly differences in operational cycles and timelines, engagement 
ranges, threat capabilities, and operating environments would be expected. 

Expeditionary warfare is a key concept. Recent Naval doctrine states “Expeditionary” implies a 
mind set, a culture, and a commitment to forces that are designed to operated forward and to 
respond swiftly.” “Expeditionary implies the ability to operate where there was previously no 
supporting infrastructure...” and “An expeditionary force is an organization tailored for a specific 
mission, vice a structure.” In essence, our naval expeditionary forces are characterized by 
forward deployment, and ability to swiftly respond to crises, a structure to build power from the sea 
when required, sustainability for long-term operations, and freedom from the need for transit or 
over flight approval from foreign governments in order to enter the scene of action. The key 
concept is one of a flexible, deployed force that can address the complete range of operations in 
support of vital national interests from peacetime presence through warfare in a fully developed 
regional conflict. 

Expeditionary warfare concepts applied to a littoral environment are used to demonstrate the 
SEER methodology and the software. The goal is to link these naval missions with their space 
support requirements. 
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Bounding the Problem . 



In stage one, the problem is bounded by considering the relationships between functional missions 
and the attributes of the systems used to accomplish these functions. The software arranges the 
relationships in a matrix where the functions (also known as the “whats” for what needs to be 
done) are on the vertical axis and the attributes (also called the “hows” for how things get done) 
are on the horizontal axis. The matrix then maps the relationships between the functions and the 
attributes (or the “whats” and the “hows”). Each square on the matrix represents a direct 
relationship between the function and the attribute. Normal practice is to identify strong, medium 
or weak direct relationships. Numerical values are assigned to the intensity of the relationship. 
Strong relationships have a high numerical value and weak relationships have a low numerical 
value. This stage can be contentious and time consuming, but when done correctly it forms the 
solid foundation upon which the rest of the analysis is performed. 

The need to properly bound the problem and reduce, where possible and appropriate, the sheer 
amount of data to be analyzed is readily apparent. 

Department of Navy (DoN) doctrine describes 82 DoN warfare tasks. Recently the DoN has 
prioritized these warfare tasks placing 26 tasks in a top priority category, 40 tasks at a middle 
priority category and the remaining tasks in a lower priority category. 

Naval Doctrine Publication 1 (NDP-1 ) describes the operational capabilities of naval expeditionary 
forces in the joint warfare environment in four separate categories: Command, Control, and 
Surveillance, Battlespace Dominance, Power Projection, and Force Sustainment. For our analysis 
it is convenient to include each of the 82 DoN tasks in one of these four lists of operation 
capabilities. 

The space force enhancement mission area is comprised of capabilities in four broad areas: 
reconnaissance and surveillance, targeting, tactical warning and attack assessment; 
communications; navigation; and environmental monitoring. Each of the four space force 
enhancement capabilities may require up to a dozen attributes to describe the function. 

The magnitude of the stage 1 task is apparent. Each of the four lists of approximately twenty 
naval operational capabilities must be evaluated against each of the four traditional space force 
enhancement mission areas. This task is even more intimidating when each of the mission area’s 
multiple attributes are considered. 
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...From the Sea 

Naval Operational Capabilities 



STAGE 1 



Power Projection 

- Precision 

- On Demand 

- All Weather 

- Etc. 

Battlespace Dominance 

- Local Sea Control 

- Local Air Control 

- Etc. 



Command, Control 
and Surveillance 

- Joint & Combined 
• Adaptable 

- Etc, 

Force Sustainment 
> Global Access 

- Extended Periods 

- Etc. 




WHATs 



HOWS 



Space System Support 




Surveillance 

- Resolution 
' Timeliness 

- Etc. 

Environmental 

- Timeliness 

- Accuracy 

- Etc. 



Navigation 

• Accuracy 

- Jam Resistance 

- Etc. 

Communications 

• Capacity 

- Timeliness 

- Etc. 



RELATIONSHIPS 

J ^ I l_ 



MATRIX APPROACH 




Identifies Naval operational 
capabilities and space support 
areas that are strongly interrelated 

• Establishes relative rankings 

• Identifies key system constraints 

• Provides numerical criteria 



Figure 2. Mission/Space Support Relationships 



Bench Markina 

Implementation of the SEER software stage 1 process automatically yields, not only an indicator of 
where the strongest relationships exist between requirements and space systems, but also a set 
of quantifiable benchmarks. 

Once complete, stage 1 ranks the attributes and identifies those with the strongest impact on the 
described functions. It also provides quantifiable benchmarks which can be used later. From this 
list, the analyst may choose those attributes with which to continue the analysis. This is how stage 
1 acts as a sieve to reduce the sheer volume of material to be considered. Figure 2 illustrates the 
stage 1 process. 



Domain Experts and SEER 

Ideally stage 1 of the SEER analysis process is completed by a group of recognized “domain 
experts.” In this case the “domain experts” are mid-grade naval warfare specialists with recent 
fleet experience. The benefits of using this type of expert in the analysis cannot be 
overemphasized. Without this expertise the result can never be vetted. 

This process allows the domain experts to work in the present. We believe it is important to focus 
the initial analysis in the present since this is the realm of requirements best understood, most 
easily and forcefully articulated and most commonly agreed upon. The SEER software stage 2 
method provides a process for projecting these quantified requirements into a future time frame. 
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Identifying Future Requirements 



Accurate descriptions of future requirements are often elusive. Stage 2 provides a process for 
projecting the previously quantified requirements into a future time frame. Identifying and 
quantifying future requirements is critical to the process of determining the relative worth of the 
various system alternatives under consideration. 

The process we have employed successfully in developing future space system requirements 
takes advantage of the fact that space force enhancement functions are accomplished to support 
terrestrial forces. Space communication, navigation, surveillance, and environmental systems are 
all intended to support terrestrial military forces. Therefore, the requirements for space systems to 
fulfill force enhancement functions can and should be derived from a careful examination of 
support requirements for new platforms, weapon systems, and tactics. 



STAGE 2 




The A in space capability due to weapon 
systems/tactical development is a “vector” in 
the direction of increased requirements. 



STAGE 1 
Relationship 
Matricee 




Key 

space systems 
capabilities for 
a particular 
Naval mission 
area 





Future Platforms, 
Tactics, & Weapons 



The A in space capability 
due to weapon systems/tactical 
development is a null vector. No 
significant change in requirements. 



+3 

^2 

♦1 

0 

-1 

-2 

-3 



Scale 

New Weapon System Is Key Driver of Quality or Quantity of Space Support 

Quality or Quantity of Space System Support increased Significantly 

Quality or Quantity of Space System Support Increased 

Quality and Quantity of Space System Support Is unchanged 

Quality or Quantity of Space System Support Is Decreased 

Quality or Quantity of Space System support Is Decreased Significantly 

New Weapon System Does not Require this Space System Capability 




The A in space capability due to weapon 
systems/tactical development is a “vector” in 
the direction of decreased requirements. 



Figure 3. Effect of New Platforms, Tactics, & Weapons 



The introduction of new platforms, weapons, and employment tactics will in many cases have a 
significant effect on the quality and quantity of support required from space systems, identifying 
and quantifying these effects as early as possible is critical to planning for the right future space 
systems. 

The effects of new platforms, weapon, and employment tactics on future space systems 
requirements can be described as a vector. A new capability requiring additional space system 
support results in a vector with a positive value. Conversely, some new capabilities may result in 
a diminished requirement for space system support and thus result in a vector with a negative 
value . Figure 3 illustrates this concept. 

Developing Measures 



172 



Developing system measures for a future time frame is a relatively straightforward process now 
based on the information we have developed in the first two stages. Two key products of stage 1 
and the output of stage 2 will be used. 

in stage 1 we developed a set of matrices showing the key relationships between naval 
operational requirements and the attributes of the systems supporting these requirements. The 
system attributes showing the strongest relationships, as determined by the normalized numerical 
scores recorded, are candidates for development as system measures. The analyst may chose to 
draw the line at any point selecting only those attributes which appear most relevant and 
discarding others. 

Recall also that stage 1 associates a benchmark value with each of the attributes. For example, a 
medium communications data rate may have been the current requirement for the attribute. This 
current communications MDR requirement would then be used as the benchmark from which the 
stage 2 vector is applied. 

If the new platforms, weapon systems, or tactics considered in stage 2 show a need for higher 
data rate communications, by exhibiting a large positive value, the analyst may chose to write a 
future MOE for HDR communications. Conversely, if the rate of change is a minus value the 
future MOE may be written as LDR. If no change was indicated (a value of near zero) then the 
requirement would continue to be for MDR communications in the future. 

Care must be taken at this stage to understand that the future MOE is only applicable to the set of 
new platforms, weapons, and tactics considered. There may be other platforms, weapons, or 
tactics driving the requirement in a different direction. 

This process of developing future measures of effectiveness is exercised as illustrated in figure 4. 

Measuring System Performance 
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Thus far, our analysis methodology has yielded a set of system performance measures directly 
traceable to the key relationships identified between Naval missions areas and space support 
functions. These performance measures have also been adjusted to account for the effects 
produced by the introduction of new platforms, weapon systems, and tactics. 

This, the fourth stage of our analysis methodology, will use the system performance measures we 
have derived to compare the capabilities of the various alternatives proposed as solutions to the 
warfighting requirements. The result of this comparison will be presented as two different figures- 
of-merit (FOM) we will use to supply values for the benefit side of the benefit-cost ratio. 

Before we describe these two figures-of-merit further, we will focus on an intermediate step to this 
solution which involves establishing relative weightings among the measures of performance. At 
this point in the analysis it is expected that the process has yielded several performance 
measures. It is also anticipated that some of these performance measures are relatively more 
important to the analysis than others. To account for this relative ranking we will use the weighting 
process illustrated in the upper left hand corner of figure 5. This process is a simple pair-wise 
comparison which involves comparing each of the performance measures with each of the other 
performance measures and deciding which is the more important. A value between 1 to 4 is used 
to score each comparison. These individual scores can be summed to form a raw numerical 
score which is then convert into a weighted category. In the example (last column) we break the 
raw scores down into three weighted categories. These MOE weightings will be carried forward to 
the System Evaluation Matrix. 
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RELATIVE MOE WEIGHTING 
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HIGH 


MOE #4 


A 


YES 


MED 


B 


NO 


HIGH 




MEASURES 


WEIGHT 


SYSTEM A 


SYSTEM B 


EXP PERF 


CONFID 


TTL VAL 


DISC VAL 


EXP PERF 


CONFID 


TTL VAL 


DISC VAL 


MOE # 1 


3 


YES 


HIGH 


30 


27 


YES 


HIGH 


30 


27 


MOE #2 


2 


YES 


MED 


20 


10 


NO 


HIGH 


0 


0 


MOE #3 


1 


YES 


MED 


10 


5 


NO 


HIGH 


0 


0 


MOE #4 


1 


YES 


MED 


10 


5 


NO 


HIGH 


0 


0 


TOTALS 








70 


47 






30 


27 


CONFIDENCE 






67.14 








90.00 







SYSTEM EVALUATION MATRIX 



Figure 5. Measuring System Performance 



The next step in our analysis process involves developing a system simulation matrix which 
describes the expected performance of each of the system alternatives in relation to each of the 
performance measures. This involves scoring each alternative in terms of its ability to fulfill the 
requirements of the individual performance measure. Each system receives a score of yes, 
partial, or no with regard to each measure. Since this score is an analysts judgment based on 
available knowledge regarding the alternative systems a confidence level is also assigned for each 
performance measure score. The confidence level is important since we are often required to 
compare the capabilities of systems at various stages of maturity. At one end of the spectrum we 
have system alternatives in the concept development phase where confidence regarding future 
performance is usually low. At the opposite end of the spectrum we have existing or deployed 
systems where extensive well documented information is available regarding performance. With 
these systems we have a high confidence in the expected performance. One can easily find 
examples at various levels in between these two extremes. This system simulation process is 
illustrated in the upper right hand corner of figure 5. 

The final step in stage 4 is development of a system evaluation matrix displaying two system 
performance figures-of-merit (FOM). A FOM is developed showing the total value and the 
discounted value of each alternative system. 

The systems’ total value is based on a simple calculation accounting for the weight assigned to 
each individual MOE and the expected performance of the system with regard to that MOE. Total 
value scores for each system alternatives are arrived at by multiplying the weight assigned to that 
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MOE by a numerical value assigned based on the expected performance with regard to that MOE. 
In the example in figure 5, if the entry with regard to expected performance is yes a value of 10 is 
used, if the entry is partial a value of 5 is used, and if the entry is no a value of 0 is used. 

A system total value score is computed by summing the value for the individual MOEs. The total 
value for each of the system alternatives can be used by a decision maker as a means of 
comparing performance of the alternatives. 



C TSTAGE 

Alternative System Total Values 
Alternative System Discounted Values 







Software options for system cost data input 

DoD categories (R&D, Procurement, O&S) 
Various formats (Single estimate, annual) 
Record uncertianty associated with cost data 

Software automatically adjusts cost data 

Computes and corrects for inflation effects 
Inflates/discounts to common value year 

Provides cost comparisons based on 

Total Lifecycle Cost (LCC) 

Average Annual Cost (ACC) 



STAGE 5B 

BENEFIT-COST ANALYSIS 
FIGURES-OF MERIT 




TOTAL VALUE 
TOTAL LIFECYCLE COST 



DISCOUNTED VALUE 
TOTAL LIFECYCLE COST 



TOTAL VALUE 
AVERAGE ANNUAL COST 



DISCOUNTED VALUE 
AVERAGE ANNUAL COST 



= FOM # 1 



= FOM #2 



= FOM #3 



= FOM #4 



Figure 6. Benefit/Cost Analysis 



The discounted value for each alternative system is arrived at by adjusting the systems total value 
score to account for the analyst confidence in his assessment of that alternatives expected 
performance. A numerical value between 0 and 1 is assigned for each confidence gradient. In 
our example, a value of 0.9 is used when confidence is high, 0.5 is used when confidence is 
medium, and 0.1 is used when confidence is low. These adjusted values are then summed to 
provide and overall discounted value for each system alternative. This second FOM provides 
another means for comparison of system performance.* 

Benefit Cost Analysis 

This final analysis stage provides a tool to develop and displaying benefit-cost comparisons. This 
stage uses stage 4 total value and discounted value figures-of-merit as entering arguments for the 
benefit side of the ratio. Data for the cost side of the ratio is included as described below. 

It should be noted that the SEER software is not a cost estimation tool. Cost data for system 
alternatives must be obtained from external sources. A variety of cost estimations models are 
available. The SEER software provides various options for inputting system cost data. 
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automatically adjusts the data where necessary, and provides a common basis for comparison of 
alternative system costs. 

Cost data input options allow the analyst to enter estimates for each of the standard DoD cost 
categories: research and development (R&D); procurement; and, operations and support (O&S). 
Estimates for these cost categories can be entered in several formats. For example, estimates 
can be entered as a single aggregated estimate of system cost for each category or the estimates 
can be entered on an annual basis for each year of the systems expected life-cycle. These 
options are necessary since it is likely that cost estimates will exist in various forms dependent 
upon the maturity of the system alternative. Systems in early concept development phases may 
have only aggregated estimates available. Systems well advanced in the development cycle 
should have detailed cost information available. To further account for the varying maturity of the 
system alternatives under consideration the analyst can also enter an assessment of the 
uncertainty associated with the cost estimates. 

The SEER software automatically adjusts the cost estimates to correct for the effects of inflation 
and inflates or discounts the data to a common value year to ensure comparability. The analyst 
enters an estimated value for inflation and selects a common value year for cost comparison. 

With these inputs and corrections factors the SEER computes the total life-cycle cost (LCC) and 
the average annual cost (ACC) for each system alternative. 

In stage 5B the SEER software develops four benefit-cost analysis figures-of-merit (FOM). These 
FOM are illustrated in figure 6. 

• FOM # 1 develops the ratio of system total value (TV) to total life-cycle cost (LCC). 

• FOM # 2 develops the ratio of system discounted value (DV) to LCC. 

• FOM # 3 develops the ratio of system TV to average annual cost (AAC). 

• FOM # 4 develops the ratio of DV to AAC. 

Each of these figures-of-merit can be used as a means for direct comparison of the worth of 
alternative systems. All of the FOM should be considered during a comparison of alternatives. 
The FOM will often provide different relative rankings depending upon whether total value or 
discounted value is in the ratio. The selection of a FOM using LCC instead of ACC can also affect 
the relative ranking of the alternative systems if the life time of the systems are significantly 
different. 

The software produces graphics which make these comparisons easier. These graphics are 
constructed so the analyst can also consider various weighting for benefit and cost. The relative 
ranking can change depending upon how heavily the analyst weights benefit versus cost.® 

Summary 

Faced with diminishing budgets, and an increasing demand to justify every dollar spent by the 
DoD on space systems, the Naval Space Command needed a tool to conduct quantitative 
evaluations of its requirements. The long held assertion that the Navy is the biggest user of space 
systems among the services needs to be elaborated in much greater detail. The Navy needed a 
tool to conduct in-depth analysis describing specific needs and assessing proposed space system 
solutions. The Aerospace Corporation, working with the Naval space Command, first developed a 
methodology and then the SEER software package to help the Command optimize its space 
investment strategy. 
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This paper shows the application of the methodology to the original space command task. In 
stage one, a set of matrices were developed to bound the example problem and provide the 
foundation of a benchmark to focus the analysis. Stage two provides a method for understanding 
how changes in the platforms, weapons, and tactics of modern Naval warfare will effect future 
requirements for space system support. Stage three provides a disciplined process for developing 
measures of effectiveness appropriate for a future time frame. The fourth stage compares the 
different system alternatives considered as solutions and produces the figures of merit for the 
benefit side of the benefit-cost ratio. The final analysis stage provides a variety of method for 
inputting cost data for alternatives systems and produces several graphic display options for 
benefit-cost comparison. 

The ability to determine and quantify the optimum mix of supporting space systems for the Naval 
warfighter drove the development of both this methodology and the companion software. The 
result, as outlined in the paper, has applications exceeding the original limited scope. The five 
stage process can be easily applied to nearly any problem where competing options can be 
compared against a baseline set of requirements. Refinements to both the methodology and the 
SEER software are planned and will be based on user applications and comments. 
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